stdlib

v7.1-rc0

STDLIB Application

Description

The STDLIB application is mandatory in the sense that the minimal system based on Erlang/OTP consists of Kernel and STDLIB. The STDLIB application contains no services.

Configuration

The following configuration parameters are defined for the STDLIB application. For more information about configuration parameters, see the app(4) module in Kernel.

shell_esc = icl | abort - Can be used to change the behavior of the Erlang shell when ^G is pressed.

restricted_shell = module() - Can be used to run the Erlang shell in restricted mode.

shell_catch_exception = boolean() - Can be used to set the exception handling of the evaluator process of Erlang shell.

shell_docs_columns = pos_integer() - Configures how wide the documentation should be rendered in the shell.

See also shell_docs:config/0.

shell_expand_location = above | below - Sets where the tab expansion text should appear in the shell. The default is below. This will open a pager below the cursor that is scrollable one line at a time with Up/Down arrow keys or 5 lines at a time with PgUp/PgDn.

shell_history_length = integer() >= 0 - Can be used to determine how many commands are saved by the Erlang shell. See edlin for more.

shell_keymap = #{} - Can be used to override the default keymap configuration for the shell.

format_shell_func = {Mod, Func} | string() | default - Can be used to set the formatting of the Erlang shell output. This has an effect on commands that have been submitted and how it is saved in history or if the formatting hotkey is pressed while editing an expression (Alt+F by default). You can specify a Mod:Func/1 that expects the whole expression as a string and returns a formatted expressions as a string. See shell:format_shell_func/1 for how to set it from inside the shell.

If instead a string is provided, it will be used as a shell command. Your command must include ${file} somewhere in the string, for the shell to know where the file goes in the command.

-stdlib format_shell_func "\"emacs -batch \${file} -l ~/erlang-format/emacs-format-file -f emacs-format-function\""

-stdlib format_shell_func "{shell, erl_pp_format_func}"

shell_prompt_func = {Mod, Func} | default - where

Mod = atom()

Func = atom()

Can be used to set a customized Erlang shell prompt function.

shell_multiline_prompt = {Mod, Func} | string() | default - where

Mod = atom()

Func = atom()

Can be used to set a customized multiline shell prompt function. The multiline prompt function takes the main prompt as its only parameter.

shell_saved_results = integer() >= 0 - Can be used to determine how many results are saved by the Erlang shell.

shell_session_slogan = string() | fun() -> string()) - The slogan printed when starting an Erlang shell. Example:

$ erl -stdlib shell_session_slogan '"Test slogan"'
Erlang/OTP 26 [DEVELOPMENT] [erts-13.0.2] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] [jit:ns]

Test slogan
1>

shell_slogan = string() | fun(() -> string()) - The slogan printed when starting the Erlang shell subsystem. Example:

$ erl -stdlib shell_slogan '"Test slogan"'
Test slogan
Eshell V13.0.2  (abort with ^G)
1>

The default is the return value of erlang:system_info(system_version).

shell_strings = boolean() - Can be used to determine how the Erlang shell outputs lists of integers.

shell_hints = boolean() - Can be used to enable/disable printing of helpful hints in the shell.

See Also

app(4), application, shell

STDLIB Release Notes

This document describes the changes made to the STDLIB application.

STDLIB 7.1

Fixed Bugs and Malfunctions

The save_module/1 command in the shell now saves both the locally defined records and the imported records using the rr/1 command.

Own Id: OTP-19647 Aux Id: GH-9816, PR-9897

It's now possible to write lists:map(fun is_atom/1, []) or lists:map(fun my_func/1, []) in the shell, instead of lists:map(fun erlang:is_atom/1, []) or lists:map(fun shell_default:my_func/1, []).

Own Id: OTP-19649 Aux Id: GH-9771, PR-9898

The shell no longer crashes when requesting to auto-complete map keys containing non-atoms.

Own Id: OTP-19659 Aux Id: PR-9896

A remote shell can now exit by closing the input stream, without terminating the remote node.

Own Id: OTP-19667 Aux Id: PR-9912

Fixed guard check for is_record/2 in the linter.

Own Id: OTP-19704 Aux Id: GH-10020, PR-10034

Improvements and New Features

Added a flag option shell_hints and function shell:hints/1. You can now disable the warning in the shell when a command is taking longer than 5 seconds.

Own Id: OTP-19759 Aux Id: PR-10121

STDLIB 7.0.3

Fixed Bugs and Malfunctions

Update PCRE2 from 10.45 to 10.46. Fixes potential buffer read overflow on regular expressions with (*scs:) and (*ACCEPT) syntax combined.

Own Id: OTP-19755 Aux Id: CVE-2025-58050

STDLIB 7.0.2

Fixed Bugs and Malfunctions

A set of small bugs in sort stability for `lists:sort/1` and `lists:keysort/1` has been fixed. The bug happened for only some, seemingly random, element sequences. Most sorts were stable.

Sort stability for `lists:sort/1` is only possible to observe when sorting lists with floating point and integer numbers of the same value.

For `lists:keysort/1` the list had to start with two tuples where the keys or the whole tuples compared equal.

Own Id: OTP-19673 Aux Id: ERIERL-1240

Fixed bug in io_lib:bformat/2 which crashed if format string contained unicode characters.

Own Id: OTP-19680 Aux Id: PR-9952

STDLIB 7.0.1

Fixed Bugs and Malfunctions

Properly strip the leading / and drive letter from filepaths when zipping and unzipping archives.

Thanks to Wander Nauta for finding and responsibly disclosing this vulnerability to the Erlang/OTP project.

Own Id: OTP-19653 Aux Id: CVE-2025-4748, PR-9941

STDLIB 7.0

Fixed Bugs and Malfunctions

Shell help now orders the commands in alphabetical order.

Own Id: OTP-19161 Aux Id: PR-8573

proc_lib:stop/1,3 (and in extension gen_server:stop/3, gen_statem:stop/3 and so on) have been updated to not throw an error if the process to be stopped exits with the same reason as given to proc_lib:stop/3.

POTENTIAL INCOMPATIBILITY

Own Id: OTP-19233 Aux Id: PR-8772

The size of an atom in the Erlang source code was limited to 255 bytes in previous releases, meaning that an atom containing only emojis could contain only 63 emojis.

While atoms are still only allowed to contain 255 characters, the number of bytes is no longer limited.

External tools that parse the AtU8 chunk of a BEAM file directly need to be updated. Tools that use beam_lib:chunks(Beam, [atoms]) to read the atom table will continue to work.

POTENTIAL INCOMPATIBILITY

Own Id: OTP-19285 Aux Id: PR-8913

argparse:help/1 now accepts unicode:chardata/0.

Own Id: OTP-19303 Aux Id: PR-8932

The literals chunk in BEAM is no longer compressed, resulting in slightly smaller BEAM files when a BEAM file is stripped using beam_lib:strip_files/1.

This is a potential incompatibility for tools that read and interpret the contents of the literal chunk. One way to update such tools to work with the new format is to retrieve the chunk using beam_lib:chunks(Beam, [literals]).

POTENTIAL INCOMPATIBILITY

Own Id: OTP-19323 Aux Id: GH-8967, PR-8988

The previous digraph_utils:preorder/1 and digraph_utils:postorder/1 did not start the traversal from root nodes. This fix makes both traversals only start or restart from a root node in one of the components, or an arbitrary node if no root node can be visited.

Own Id: OTP-19393 Aux Id: PR-9171

Auto-completion in the shell is now significantly faster for function parameters that uses complex custom types.

Own Id: OTP-19413 Aux Id: PR-9271

Stringfying a non-latin1 atom will now produce a readable string instead of encoding each character using \x{...} escape sequences. Example:

-define(S(T), ??T).

atom() ->
    ?S('атом').

The atom/0 function now returns "'атом'" instead of "'\\x{430}\\x{442}\\x{43E}\\x{43C}'".

Own Id: OTP-19421 Aux Id: GH-9173, PR-9276

A few minor issues were corrected in m:syntax_tools, as well in the erl_anno module.

Own Id: OTP-19422 Aux Id: PR-9253

dets could print error messages to standard output when repairing DETS files. This has been changed to send the messages to logger.

ets:fun2ms would print an error message to standard output as well as returning an error tuple. The printing of the message has been removed.

Own Id: OTP-19427 Aux Id: PR-9232, PR-9446

The functions for converting to and from the RFC1339 date and time format would not properly handle fractional seconds for negative times.

Own Id: OTP-19441 Aux Id: GH-9279, PR-9280

Replaced calls to deprecated crypto:start() with application:start(crypto).

Own Id: OTP-19485 Aux Id: PR-8592

Fixed a bug when calling shell completion on a reserved word followed by a ( would crash the shell.

Own Id: OTP-19511 Aux Id: GH-9470

Corrected the spec of ets:update_element/4.

Own Id: OTP-19514 Aux Id: PR-9504

Corrected the spec for ets:info/1.

Own Id: OTP-19515 Aux Id: PR-9514

Fixed crash when defining records with a string field in the shell

Own Id: OTP-19533 Aux Id: GH-9557

Details in the hibernation implementation and time-out handling has been improved for gen_statem. In particular to avoid selective receive when cancelling a time-out.

Own Id: OTP-19540 Aux Id: PR-9579

Fixed a bug when getting help on a module compiled without debug_info.

Own Id: OTP-19583 Aux Id: PR-9654

Fix zip extraction to wrap invalid DOS timestamps to their correct value instead of returning the actual value. Before this fix the timestamp returned could have a second greater than 59. The bug has been present since Erlang/OTP 27.1.

Own Id: OTP-19593 Aux Id: PR-9537, GH-9536

Enhance specs of timeout for improving documentation and dialyzer analysis.

Own Id: OTP-19604 Aux Id: PR-9574

Improvements and New Features

Singleton type variables in an union type do not make sense from Dialyzer's point of view. The following example is ill-typed:

-spec run_test(Opts) -> term()
      when Opts :: {join_specs, Bool} | {test, Bool}.

This used to be reported as a warning. In OTP-28, this is an error

Own Id: OTP-19125 Aux Id: PR-8556

By default, sets created by the sets module will now be represented as maps.

Own Id: OTP-19127 Aux Id: PR-8429

For various error types, the compiler now tries to suggest potential fixes by adding "did you mean ...?" at the end of error messages.

When a function is used with wrong arity, the compiler will try to suggest a defined function with the same name but a different arity. For example, given the following module:

-module(typos).
-export([t/0]).
bar(A) -> A.
bar(A,A,A) -> A.
bar(A,A,A,A) -> A.
t() -> bar(0, 0).

The compiler will emit the following message:

typo.erl:6:12: function bar/2 undefined, did you mean bar/1,3,4?
%   6|     t() -> bar(0, 0).
%    |            ^

For compiler errors that can easily be caused by typos, the compiler will try to suggest what the correct variable or function name, could be. For example, given the following module:

-module(typos).
-export([bar/2]).

bar(A0, B0) ->
    A + B.

the compiler will emit the following error messages:

typos.erl:5:5: variable 'A' is unbound, did you mean 'A0'?
%    5|     A + B.
%     |     ^

typos.erl:5:9: variable 'B' is unbound, did you mean 'B0'?
%    5|     A + B.
%     |         ^

Error types that now suggest correct arities: bad_inline, undefined_nif, bad_nowarn_unused_function, bad_nowarn_bif_clash, undefined_function.

Error types that now suggest correct names: bad_inline, undefined_nif, bad_nowarn_unused_function, undefined_on_load, undefined_function, undefined_record, undefined_field, unbound_var.

Using a function with wrong arity has higher precedence than having a typo in the function name. If the compiler can find a defined function with the same name but a different arity, it will not suggest a defined function with a close-enough name, regardless of arity.

Own Id: OTP-19180 Aux Id: PR-8699, PR-9094

Comprehensions have been extended with zip generators according to EEP 73.

Example:

1> [A+B || A <- [1,2,3] && B <- [4,5,6]].
[5,7,9]

Own Id: OTP-19184 Aux Id: PR-8926

Before restarting a child, a supervisor must check if the restart limit is reached. This adds a penalty to the overall restart time, which should be kept low. The algorithm has been optimized from 2*O(n) to O(n) behavior.

Own Id: OTP-19204 Aux Id: PR-8261

Added the possibility to configure shell docs column width through the stdlib parameter shell_docs_columns.

Own Id: OTP-19224 Aux Id: PR-8651

The io:setopts/2 function now accepts the line_history option for more explicit handling of when to save shell history.

Own Id: OTP-19230 Aux Id: PR-8792

The shell now prints a help message explaining how to interrupt a running command when stuck executing a command for longer than 5 seconds.

Own Id: OTP-19231 Aux Id: PR-8793

Binaries can now be used as input to calendar:rfc3339_to_system_time/2, and produced as output of calendar:system_time_to_rfc3339/2.

Own Id: OTP-19250 Aux Id: PR-8812

The erl -noshell mode has been updated to have two sub modes called raw and cooked, where cooked is the old default behaviour and raw can be used to bypass the line-editing support of the native terminal. Using raw mode it is possible to read keystrokes as they happen without the user having to press Enter. Also, the raw mode does not echo the typed characters to stdout. An example of how to create a tic-tac-toe game using this mechanism is included in the documentation.

Own Id: OTP-19314 Aux Id: PR-8962, GH-8037

Added io:get_password/0 that can read passwords from stdin when in "raw" -noshell mode.

Own Id: OTP-19315 Aux Id: PR-8962, PR-9006

New strict generators have been added for comprehensions.

The currently existing generators are "relaxed": they ignore terms in the right-hand side expression that do not match the left-hand side pattern.

The new strict generators fail with exception badmatch if a pattern doesn't match.

Examples:

Using the current relaxed generator operator <-, any element not matching the pattern {_,_} will be silently discarded:

1> [T || {_,_}=T <- [{ok,1},ok,{error,2}]].
[{ok,1},{error,2}]

If the intention is that all lists processed by a list comprehension must only contain tuples of size two, using the new strict version of the operator ensures that term not matching will cause a crash:

2> [T || {_,_}=T <:- [{ok,1},ok,{error,2}]].
** exception error: no match of right hand side value ok

Using the strict generator operator to mark the intention that all list elements must match the pattern could help finding mistakes quicker if something unpexected is added to the list processed by the generator.

The strict version for bitstring generators is <:=.

Own Id: OTP-19317 Aux Id: PR-8625

New options for suppressing behaviour warnings have been added:

nowarn_conflicting_behaviours

nowarn_undefined_behaviour_func

nowarn_undefined_behaviour

nowarn_undefined_behaviour_callbacks

nowarn_ill_defined_behaviour_callbacks

nowarn_ill_defined_optional_callbacks

Own Id: OTP-19334 Aux Id: GH-8985, PR-9020

The join(Binaries, Separator) function that joins a list of binaries has been added to the binary module.

Own Id: OTP-19337 Aux Id: GH-8099, PR-8100

The supervisor:which_child/2 function has been added to facilitate getting the pid of a sibling process; that is a process under same supervisor as the process that calls to call the new function.

Own Id: OTP-19345 Aux Id: PR-8976

The function erl_anno:set_end_location/2 for setting the end location of a token has been added.

Own Id: OTP-19354 Aux Id: PR-8966

Added a warning for calling non-exported functions with the remote function call syntax from the same module, and likewise for the remote fun syntax.

Own Id: OTP-19371 Aux Id: GH-9092, PR-9095

The warn_deprecated_catch option enables warnings for use of old-style catch expressions on the form catch Expr instead of the modern try ... catch ... end. To prevent new uses of uses of old catches to be added, this compiler option can be enabled on the project level and -compile(nowarn_deprecated_catch). added to individual files that still contain old catches.

Own Id: OTP-19425 Aux Id: PR-9154

Module re has been updated to use PCRE2, which is mostly backward compatible with PCRE.

The most noticeable incompatibilities are

The default character encoding is pure ASCII and not Latin1. Unicode support is still available with options unicode and ucp.

Options bsr_anycrlf, bsr_unicode and {newline,_} are only set when a regex is compiled and cannot be changed at matching for precompiled regex.

POTENTIAL INCOMPATIBILITY

Own Id: OTP-19431 Aux Id: PR-9299, PR-9610

Defining a fun in terms of an imported function is not allowed. Before this release, the compiler would not catch this kind of error if the name of the imported function happened to be a BIF. Consider this example:

-module(fun_example).
-export([foo/0, bar/0]).
-import(m, [max/2, not_a_bif/0]).

foo() ->
    fun max/2.

bar() ->
    fun not_a_bif/0.

The compiler in Erlang/OTP 27 would generate the following messages:

fun_example.erl:9:5: function not_a_bif/0 undefined
%    9|     fun not_a_bif/0.
%     |     ^

fun_example.erl:3:2: Warning: import directive overrides auto-imported BIF max/2 --
use "-compile({no_auto_import,[max/2]})." to resolve name clash
%    3| -import(m, [max/2, not_a_bif/0]).
%     |  ^

That is, there would be a (cryptic) error for fun not_a_bif/0, but only a warning for fun max/2.

When compiling with this release, both attempts to create a fun will result in error messages (as well as a warning):

fun_example.erl:6:5: creating a fun from imported name max/2 is not allowed
%    6|     fun max/2.
%     |     ^

fun_example.erl:9:5: creating a fun from imported name not_a_bif/0 is not allowed
%    9|     fun not_a_bif/0.
%     |     ^

fun_example.erl:3:2: Warning: import directive overrides auto-imported BIF max/2 --
use "-compile({no_auto_import,[max/2]})." to resolve name clash
%    3| -import(m, [max/2, not_a_bif/0]).
%     |  ^

Also, attempting to call a local function having the same name as auto-imported BIF would result in an error if the BIF was added to Erlang/OTP before R14, and a warning for newer BIFs. This has been changed to always emit a warning. For example:

-module(bif_example).
-export([bar/1]).

bar(B) ->
    is_boolean(B).

is_boolean(B) ->
        B =:= true orelse B =:= false.

will now result in the following warning instead of an error:

if_example.erl:5:5: Warning: ambiguous call of overridden auto-imported BIF is_boolean/1 --
use erlang:is_boolean/1 or "-compile({no_auto_import,[is_boolean/1]})." to resolve name clash
%    5|     is_boolean(B).
%     |     ^

Own Id: OTP-19432 Aux Id: PR-9246

It is now possible to use any base for floating point numbers as described in EEP 75: Based Floating Point Literals.

Computers represent floating point numbers in binary, but such numbers are typically printed using base ten, for example 0.314159265e1. To maintain exact bit-level precision when converting numbers to and from text, it is better to use a base that matches the internally used base, such as 16 for a compact but still exact representation, or 2 for visualizing or writing down the exact internal format. One particular case where such exact representations are useful is in code generating tools.

Examples:

> 2#0.111.
0.875
> 16#fefe.fefe#e16.
1.2041849337671418e24

Own Id: OTP-19452 Aux Id: PR-9106

The callback function handle_continue/2 in gen_server callback modules is now cached like the others, thanks to code cleanup and optimization of the internal behaviour loop.

This should only improve performance, not affect functionality.

Own Id: OTP-19474 Aux Id: PR-9333

Encoding done by the json module has been optimized.

Own Id: OTP-19476 Aux Id: PR-9251

There is a new zstd module that does Zstandard compression.

Own Id: OTP-19477 Aux Id: PR-9316

Fixed licenses in files and added ORT curations to the following apps: otp, eldap, erl_interface, eunit, parsetools, stdlib, syntax_tools, and ERTS.

Own Id: OTP-19478 Aux Id: PR-9376, PR-9402, PR-9819

Functions of a module can now be grouped in the shell code completion by using the group key in the -doc attribute e.g. -doc(#{group=><<"Public API">>). fetch()->....

Functions, callbacks and types in the module reference documentation of OTP is now grouped using this feature.

Own Id: OTP-19483 Aux Id: PR-9408

Added calendar:universal_time_to_system_time/1,2 and calendar:local_time_to_system_time/1,2

Own Id: OTP-19505 Aux Id: PR-9445

Improve error messages for json:decode/1.

Own Id: OTP-19508 Aux Id: PR-9484

ETS heir can be set without getting an ETS-TRANSFER message. Useful when the heir is a supervisor process that cannot handle custom messages.

Own Id: OTP-19512 Aux Id: PR-7970

Added support for the Unicode 16 standard.

Own Id: OTP-19516 Aux Id: PR-9518, PR-9141

When documenting a function or type that needs to deal with durations, usually we can document it as "time in milliseconds". Since the timer family of functions (hms, hours, seconds, ...) all return time in milliseconds, it is useful to be able to use this type in type specifications.

Own Id: OTP-19526 Aux Id: PR-9515

A new event time-out has been implemented in gen_server, that behaves more like the one in gen_statem.

See the type gen_server:action/0 for {timeout|hibernate,...}, and also related functions.

Own Id: OTP-19537 Aux Id: PR-9287, PR-9615, PR-9621

Line numbers used to be reported in the following way:

1> lists:last([]).
** exception error: no function clause matching lists:last([]) (lists.erl, line 389)

Starting from Erlang/OTP 28, line numbers are now reported in the following way:

1> lists:last([]).
** exception error: no function clause matching lists:last([]) (lists.erl:389)

Own Id: OTP-19538 Aux Id: PR-9468

Upgrade pcre2 to 10.45

Own Id: OTP-19541 Aux Id: PR-9582

Added functions that produce utf-8 binaries instead of iolists. New functions are: io_lib:bformat/2, io_lib:bformat/3, io_lib:bfwrite/2, io_lib:bfwrite/3, io_lib:bwrite/2 and io_lib:bwrite_string/3.

Own Id: OTP-19556 Aux Id: PR-9772

The license and copyright header has changed format to include an SPDX-License-Identifier. At the same time, most files have been updated to follow a uniform standard for license headers.

Own Id: OTP-19575 Aux Id: PR-9670

A list of PCRE2 incompatibilities is documented in a user's guide for stdlib.

Own Id: OTP-19578 Aux Id: PR-9705

Change automatic hibernation of static supervisors so that they will hibernate after being idle for 1 second instead of only after starting, dynamic supervisors (simple_one_for_one) will not be hibernated at all. An option to the supervisor is added to make it configurable for the application. This option defaults to 1 second for static supervisors and to infinity for the simple_one_for_one supervisors.

POTENTIAL INCOMPATIBILITY

Own Id: OTP-19597 Aux Id: PR-9680

STDLIB 6.2.2.2

Fixed Bugs and Malfunctions

A set of small bugs in sort stability for `lists:sort/1` and `lists:keysort/1` has been fixed. The bug happened for only some, seemingly random, element sequences. Most sorts were stable.

Sort stability for `lists:sort/1` is only possible to observe when sorting lists with floating point and integer numbers of the same value.

For `lists:keysort/1` the list had to start with two tuples where the keys or the whole tuples compared equal.

Own Id: OTP-19673 Aux Id: ERIERL-1240

STDLIB 6.2.2.1

Fixed Bugs and Malfunctions

The save_module/1 command in the shell now saves both the locally defined records and the imported records using the rr/1 command.

Own Id: OTP-19647 Aux Id: GH-9816, PR-9897

It's now possible to write lists:map(fun is_atom/1, []) or lists:map(fun my_func/1, []), in the shell, instead of lists:map(fun erlang:is_atom/1, []) or lists:map(fun shell_default:my_func/1, []).

Own Id: OTP-19649 Aux Id: GH-9771, PR-9898

Properly strip the leading / and drive letter from filepaths when zipping and unzipping archives.

Thanks to Wander Nauta for finding and responsibly disclosing this vulnerability to the Erlang/OTP project.

Own Id: OTP-19653 Aux Id: CVE-2025-4748, PR-9941

Shell no longer crashes when requesting to autocomplete map keys containing non-atoms.

Own Id: OTP-19659 Aux Id: PR-9896

A remote shell can now exit by closing the input stream, without terminating the remote node.

Own Id: OTP-19667 Aux Id: PR-9912

STDLIB 6.2.2

Fixed Bugs and Malfunctions

Fixed crash when fetching initial_call when user code have modified the process_dictionary.

Own Id: OTP-19546 Aux Id: ERIERL-1205, PR-9596

STDLIB 6.2.1

Fixed Bugs and Malfunctions

Fixed argparse:help/2 to accept the program name as part of the command path.

Own Id: OTP-19397 Aux Id: PR-9160

Fixed argparse:format_help/2 crash on 'hidden' command.

Own Id: OTP-19400 Aux Id: PR-9151, GH-9150

Fixed the type specification for timer:sleep/1 by adding the value infinity to its input type.

Own Id: OTP-19442 Aux Id: PR-9303

Eliminated a crash in zip:unzip/1 while unzipping an archive where a directory within was read-only. This bug was introduced in Erlang/OTP 27.1.

Own Id: OTP-19447 Aux Id: GH-9332, PR-9335

Fixed map comprehension result when a key value is replaced.

Own Id: OTP-19459 Aux Id: GH-9348, PR-9358

Fixed string:jaro_similarity/1 for matching strings of length 1.

Own Id: OTP-19468 Aux Id: PR-9371

STDLIB 6.2

Fixed Bugs and Malfunctions

Made it possible to expand help text displayed by pressing ^[h by pressing ^[h again.

Own Id: OTP-19260 Aux Id: PR-8884

Defining a fun in the shell using the syntax fun Name/Arity would fail. This has been corrected so that the following now works:

1> F = fun is_atom/1.
#Fun.erl.42.18682967>
> F(a).
true
3> Id = fun id/1.
#Fun.erl.42.18682967>
4> Id(42).
** exception error: undefined shell command id/1
5> id(I) -> I.
ok
6> Id(42).
42

The Debugger has also been corrected to correctly handle this syntax for a BIF.

Own Id: OTP-19322 Aux Id: GH-8963, PR-8987

Fixed a bug where completion of 'fun(' would cause the shell to crash.

Own Id: OTP-19351 Aux Id: PR-9043

Fixed a bug causing the shell to crash while trying to complete an expression starting with a '/' or a variable followed by '(' or '/'. E.g. Foo/ and Foo(.

Own Id: OTP-19361 Aux Id: PR-9078

zip:extract/2 with keep_old_files now respects the cwd option.

Own Id: OTP-19370 Aux Id: PR-9097, GH-9087

Fixed an error in uri_string:percent_decode spec

Own Id: OTP-19380 Aux Id: GH-8755

Improvements and New Features

Updated shell docs to display the type spec, that is, h(erlang, min, 2)) now prints the type spec and documentation in the shell.

> h(erlang,min,2).

  -spec min(Term1, Term2) -> Minimum
               when Term1 :: term(), Term2 :: term(), Minimum :: term().

  Returns the smallest of Term1 and Term2. If the terms compare equal with the == operator, Term1 is returned.

Own Id: OTP-19234 Aux Id: GH-8544, PR-8833

The file:io_device/0 type has been updated to clearly show the difference between a raw and cooked IoDevice.

Own Id: OTP-19301 Aux Id: PR-8956

Added json:format_key_value_list/3 and json:format_key_value_list_checked/3.

Own Id: OTP-19320 Aux Id: PR-8889

Improved documentation of timers.

Own Id: OTP-19360 Aux Id: ERIERL-1149, PR-9062

Added logging support to io:user/0, io:standard_io/0 and io:standard_error/0. See io:setopts/2 for more details.

Own Id: OTP-19372 Aux Id: PR-8947

STDLIB 6.1.2

Fixed Bugs and Malfunctions

With this change, uri_string:normalize assumes empty path (do not crash) when no path is provided in the URI map.

Own Id: OTP-19266 Aux Id: ERIERL-1127, PR-8890

Fixed spec for json:format/3.

Own Id: OTP-19286 Aux Id: GH-8880, PR-8914

STDLIB 6.1.1

Fixed Bugs and Malfunctions

Remove whitespace stripping of returned binaries in json:decode/3.

Own Id: OTP-19227 Aux Id: ERIERL-1130, PR-8809

Fix zip:unzip/2 to not crash when extracting zip files with garbage in the Zip64 extra header. This bug was introduced in Erlang 27.1 and has so far only been seen on some archives creates by MS Excel.

Own Id: OTP-19241 Aux Id: PR-8836

With this change, shutdown procedure handles a race condition between supervisor executing a shutdown and child process termination from other reason.

Own Id: OTP-19256 Aux Id: PR-8780

STDLIB 6.1

Fixed Bugs and Malfunctions

The help printout for incorrect io:format/0 strings now handles the k modifier correctly.

Own Id: OTP-19146 Aux Id: PR-8611, GH-8568

Fixed a bug that caused the shell completion to crash when keyword and tuple appeared on the same line.

Own Id: OTP-19157 Aux Id: PR-8638

Due to PR-7419/OTP-18671, the cached internal value of the callback_mode started leaking out to logger reports, which could cause logger handlers to crash. This has now been fixed to show the value that was set, as before caching.

Own Id: OTP-19164 Aux Id: GH-8605, PR-7419, OTP-18671

Fixed an emulator crash relating to compressed ETS tables.

Own Id: OTP-19176 Aux Id: PR-8683

The error description for maps:update/3 will no longer insist that the third argument is not a map when a key could not be found

Own Id: OTP-19189

Multiple issues have been corrected in the markdown parser that creates documentation for the shell.

The parser was incorrectly parsing formatted markdown (either bold or italics) within parenthesis. This used to not be shown correctly in the shell documentation (_Option._), which was displayed verbatim. This fix makes Option. to appear in italics.

The markdown parser is also used in the creation of other documentation formats, so this was a bug that affected other generated documentation formats.

Own Id: OTP-19200 Aux Id: GH-8738, PR-8739

Fixed category for some codepoint ranges in unicode_util.

Own Id: OTP-19210 Aux Id: GH-8748

Fixed argparse to print sub-commands help when available.

Own Id: OTP-19222 Aux Id: PR-8777

Improvements and New Features

Class annotation to HTML from fenced blocks have been added.

Own Id: OTP-19105 Aux Id: PR-8499

Added JSON formatting functions for indented output.

Own Id: OTP-19112

Improved illegal pattern error for accidental map associations.

Own Id: OTP-19128 Aux Id: PR-8555

Progress reports for a dynamically started supervisor will now be logged at debug level.

POTENTIAL INCOMPATIBILITY

Own Id: OTP-19202 Aux Id: PR-8261, GH-8715, PR-8741

The zip module has been updated with support for:

zip64 archives - Archives larger than 4GB or with more than 2^32 entries.

extended timestamps - Higher resolution and in UTC.

UID/GID - Save and extract the original UID/GID.

Fixes so that permission mode attributes are correctly read and set for files in archives.

zip:list_dir/2 now also returns directories, not only files. (You can disable this behaviour by using the option skip_directories).

Various bugs in the original implementation have also been fixed, such as:

Correctly encode and decode the DOS timestamps for entries within an archive (that is the non-extended timestamp).

Fix DOS timestamps to be set to localtime instead of UTC (use extended timestamps for UTC timestamps).

Use the unix file attributes read from disk when creating archives instead of setting everything to 644.

Own Id: OTP-19214 Aux Id: PR-8765

STDLIB 6.0.1

Fixed Bugs and Malfunctions

Fix so that missing -doc({file, File}) files only result in a warning and not an error.

Own Id: OTP-19099 Aux Id: PR-8542

Fixed json bugs, json:encode_key_value_list/2 did not generate arrays and json:decode/3 did not invoke the user callback for 0.

Own Id: OTP-19106 Aux Id: PR-8581, GH-8580, PR-8519

STDLIB 6.0

Fixed Bugs and Malfunctions

The specs in module binary has been updated to reflect what is allowed by the documentation.

Own Id: OTP-18684 Aux Id: PR-7481

Several functions in the binary module would accept arguments of the wrong type under certain circumstances. In this release, they now raise an exception when incorrect types are given.

The following functions would accept an invalid pattern if the subject binary was empty or if the {scope,{0,0}} option was given: binary:match/2,3, binary:matches/2,3, binary:replace/3,4, and binary:split/2,3

The call binary:copy(<<1:1>>, 0) would return an empty binary instead of raising an exception. Similarly, calls to binary:part/2,3 attempting to extract 0 bytes at position 0 of a bitstring would return an empty binary instead of raising an exception.

Own Id: OTP-18743 Aux Id: PR-7607, PR-7628

The documentation for the preprocessor now mentions that defined(Name) can be called in the condition for an -if or -elif directive to test whether Name is the name of a defined macro. (This feature was implemented in OTP 21.)

If a function call in an -if or -elif with a name that is not the name of a guard BIF, there would not be a compilation error, but would instead cause the lines following the directive to be skipped. This has now been changed to be a compilation error.

POTENTIAL INCOMPATIBILITY

Own Id: OTP-18784 Aux Id: GH-7706, PR-7726

get_until requests using the I/O protocol now correctly return a binary or list when eof is the last item returned by the callback.

Own Id: OTP-18930 Aux Id: PR-7993, GH-4992

The error handling the simple_one_for_one supervisor has been enhanced. A transient child returning ignore will no longer cause a crash.

Also, automatic shutdown has been disabled because it does not make sense for this supervisor type. That is was allowed is considered a bug. Therefore, we don't consider this an incompatible change.

POTENTIAL INCOMPATIBILITY

Own Id: OTP-19029 Aux Id: PR-8230

Fix shell expansion to not crash when expanding a map with non-atom keys and to not list zero arity functions when an argument has been given.

Own Id: OTP-19073 Aux Id: PR-8375, GH-8366, GH-8365, GH-8364

Improvements and New Features

The functions is_equal/2, map/2, and filtermap/2 have been added to the modules sets, ordsets, and gb_sets.

Own Id: OTP-18622 Aux Id: PR-7183, PR-7232

The compiler now emits nicer error message for function head mismatches. For example, given:

a() -> ok;
a(_) -> error.

Erlang/OTP 26 and earlier would emit a diagnostic similar to:

t.erl:6:1: head mismatch
%    6| a(_) -> error.
%     | ^

while in Erlang/OTP 27 the diagnostic is similar to:

t.erl:6:1: head mismatch: function a with arities 0 and 1 is regarded as two distinct functions. Is the number of arguments incorrect or is the semicolon in a/0 unwanted?
%    6| a(_) -> error.
%     | ^

Own Id: OTP-18648 Aux Id: PR-7383

zip:create/2,3 will now tolerate POSIX timestamps in the provided file_info records.

Own Id: OTP-18668

The callback function gen_statem:handle_event/4 has been cached in the gen_statem engine to optimize callback call speed.

Own Id: OTP-18671 Aux Id: PR-7419

The type beam_lib:beam/0 is now exported.

Own Id: OTP-18716 Aux Id: PR-7534

The documentation for the binary module has been improved.

Own Id: OTP-18741 Aux Id: PR-7585

binary:replace/3,4 now supports using a fun for supplying the replacement binary.

Own Id: OTP-18742 Aux Id: PR-7590

Triple-Quoted Strings has been implemented as per EEP 64. See String in the Reference Manual.

Example:

1> """
   a
   b
   c
   """.
"a\nb\nc"

Adjacent string literals without intervening white space is now a syntax error, to avoid possible confusion with triple-quoted strings. For example:

1> "abc""xyz".
"xyz".
* 1:6: adjacent string literals without intervening white space

POTENTIAL INCOMPATIBILITY

Own Id: OTP-18750 Aux Id: OTP-18746, PR-7313, PR-7451

The new function proc_lib:set_label/1 can be used to add a descriptive term to any process that does not have a registered name. The name will be shown by tools such as c:i/0, observer, and it will be included in crash reports produced by processes using gen_server, gen_statem, gen_event, and gen_fsm.

The label for a process can be retrieved by calling proc_lib:get_label/1.

Note that those functions work on any process, not only processes that use proc_lib.

Example:

1> self().
<0.90.0>
2> proc_lib:set_label(my_label).
ok
3> i().
    .
    .
    .
<0.90.0>              erlang:apply/2                        2586    75011    0
my_label              c:pinfo/2                               51
4> proc_lib:get_label(self()).
my_label

Own Id: OTP-18789 Aux Id: PR-7720, PR-8003

-callback attributes has been added to modules sys and erl_error.

Own Id: OTP-18793 Aux Id: PR-7703

Several new functions that accept funs have been added to module timer.

Functions apply_after/2, apply_interval/2, and apply_repeatedly/2 accept a nullary fun as the second argument, while functions apply_after/3, apply_interval/3, and apply_repeatedly/3 accept an n-ary fun as the second and a list of n arguments for the fun as the third argument.

Own Id: OTP-18808 Aux Id: PR-7649

Sigils on string literals have been implemented as per EEP 66, that is: binary and string sigils in verbatim and escape characters variants, as well as a default (vanilla) Sigil. All for ordinary strings and for triple-quoted strings (EEP 64). See Sigils in the Reference Manual.

Examples:

1> ~"Björn".
<<"Björn"/utf8>>
2> ~b"Björn".
<<"Björn"/utf8>>
3> ~S"\s*(\w+)".
"\\s*(\\w+)"
4> ~B"\s*(\w+)".
<<"\\s*(\\w+)">>

Own Id: OTP-18825 Aux Id: OTP-18750, PR-7684

Functions shell:default_multiline_prompt/1, shell:inverted_space_prompt/1, and shell:prompt_width/1 have been exported to help with custom prompt implementations.

Own Id: OTP-18834 Aux Id: PR-7675, PR-7816

The shell now pages long output from the documentation help command (h(Module)), auto completions and the search command.

Own Id: OTP-18846 Aux Id: PR-7845

The M-h hotkey (Alt/Option-h) now outputs help for the module or function directly before the cursor.

Own Id: OTP-18847 Aux Id: PR-7846

Added support for adding a custom code formatter that formats your multi-line shell commands in your preferred formatting on submission. See shell:format_shell_func/ and shell:erl_pp_format_func/1.

Own Id: OTP-18848 Aux Id: PR-7847

Added shell functions for viewing, forgetting and saving locally defined functions, types and records.

Own Id: OTP-18852 Aux Id: PR-7844

Added string:jaro_similarity/2, which can be used to calculate the similarity between two strings.

Own Id: OTP-18865 Aux Id: PR-7879

The new function ets:update_element/4 is similar to ets:update_element/3, but takes a default tuple as the fourth argument, which will be inserted if no previous record with that key exists.

Own Id: OTP-18870 Aux Id: PR-7857

Added functions to retrieve the next higher or lower key/element from gb_trees and gb_sets, as well as returning iterators that start at given keys/elements.

Own Id: OTP-18874 Aux Id: PR-7745

When the shell built-in function c/1,2 is used to re-compile a module, the current working directory of the original compilation is now added to the include path.

Own Id: OTP-18908 Aux Id: PR-7957

The timer module now uses a private table for its internal state, slightly improving its performance.

Own Id: OTP-18914 Aux Id: PR-7973

EEP-59 - Documentation Attributes has been implemented.

Documentation attributes can be used to document functions, types, callbacks, and modules. The keyword -moduledoc "Documentation here". is used to document modules, while -doc "Documentation here". can be used on top of functions, types, and callbacks to document them, respectively.

Types, callbacks, and function documentation can be set to hidden either via -doc false or -doc hidden. When documentation attributes mark a type as hidden, they will not be part of the documentation.

The documentation from moduledoc and doc gets added by default to the binary beam file, following the format of EEP-48.

Using the compiler flag warn_missing_doc will raise a warning when -doc attributes are missing in exported functions, types, and callbacks.

Using the compiler flag warn_missing_spec_documented will raise a warning when spec attributes are missing in documented functions, types, and callbacks.

moduledocs and docs may refer to external files to be embedded, such as -doc {file, "README.md"}., which refers to the file README.md found in the current working directory.

The compiler warns about exported functions whose specs refer to hidden types. Thus, there will be warnings when a hidden type (meaning, the type is not part of the documentation) gets used in an exported function.

Own Id: OTP-18916 Aux Id: PR-7936

New ets functions ets:first_lookup/1, ets:next_lookup/2, ets:prev_lookup/2 and ets:last_lookup/1. Example: ets:next_lookup/1 is equivalent to ets:next/2 followed by ets:lookup/2 with the next key. The new combined functions are more efficient and with guaranteed atomicity.

Own Id: OTP-18923 Aux Id: PR-6791

The maybe expression is now enabled by default.

To use maybe as an atom, it needs to be single-quoted. Alternatively, the maybe expression can be disabled by disabling the maybe_expr feature. That can be done by placing the following the line at the beginning of an Erlang source file:

-feature(maybe_expr, disable).

Another way to disable the maybe_expr feature is by passing the -disable-feature option to erlc:

erlc -disable-feature maybe_expr some_file.erl

Own Id: OTP-18944 Aux Id: PR-8067

The compiler will now raise a warning when updating record/map literals. As an example, consider this module:

-module(t).
-export([f/0]).
-record(r, {a,b,c}).

f() ->
    #r{a=1}#r{b=2}.

The compiler raises the following warning:

1> c(t).
t.erl:6:12: Warning: expression updates a literal
%    6|     #r{a=1}#r{b=2}.
%     |            ^

Own Id: OTP-18951 Aux Id: PR-8069

The documentation has been migrated to use Markdown and ExDoc.

Own Id: OTP-18955 Aux Id: PR-8026

Optimized ets:foldl and ets:foldr to use new ets:next_lookup. Also made them immune against table renaming.

Own Id: OTP-18993 Aux Id: PR-8048

Windows now supports all functions in math.

Own Id: OTP-19001 Aux Id: PR-8164

erl_lint (and by extension the compiler) will now warn for code using deprecated callbacks.

The only callback currenly deprecated is format_status/2 in gen_server, gen_event and gen_statem.

You can use nowarn_deprecated_callback to silence the warning.

Own Id: OTP-19010 Aux Id: PR-8205

There is a new module json for encoding and decoding JSON.

Both encoding and decoding can be customized. Decoding can be done in a SAX-like fashion and handle multiple documents and streams of data.

Own Id: OTP-19020 Aux Id: PR-8111

STDLIB 5.2.3.5

Fixed Bugs and Malfunctions

A set of small bugs in sort stability for `lists:sort/1` and `lists:keysort/1` has been fixed. The bug happened for only some, seemingly random, element sequences. Most sorts were stable.

Sort stability for `lists:sort/1` is only possible to observe when sorting lists with floating point and integer numbers of the same value.

For `lists:keysort/1` the list had to start with two tuples where the keys or the whole tuples compared equal.

Own Id: OTP-19673 Aux Id: ERIERL-1240

STDLIB 5.2.3.4

Fixed Bugs and Malfunctions

It's now possible to write lists:map(fun is_atom/1, []) or lists:map(fun my_func/1, []), in the shell, instead of lists:map(fun erlang:is_atom/1, []) or lists:map(fun shell_default:my_func/1, []).

Own Id: OTP-19649 Aux Id: GH-9771 PR-9898

Properly strip the leading / and drive letter from filepaths when zipping and unzipping archives.

Thanks to Wander Nauta for finding and responsibly disclosing this vulnerability to the Erlang/OTP project.

Own Id: OTP-19653 Aux Id: CVE-2025-4748 PR-9941

A remote shell can now exit by closing the input stream, without terminating the remote node.

Own Id: OTP-19667 Aux Id: PR-9912

STDLIB 5.2.3.3

Fixed Bugs and Malfunctions

Fixed an error in uri_string:percent_decode spec

Own Id: OTP-19380 Aux Id: GH-8755

STDLIB 5.2.3.2

Fixed Bugs and Malfunctions

With this change, shutdown procedure handles a race condition between supervisor executing a shutdown and child process termination from other reason.

Own Id: OTP-19256 Aux Id: PR-8780

With this change, uri_string:normalize assumes empty path (do not crash) when no path is provided in the URI map.

Own Id: OTP-19266 Aux Id: ERIERL-1127, PR-8890

STDLIB 5.2.3.1

Fixed Bugs and Malfunctions

Fixed a bug that caused the shell completion to crash when keyword and tuple appeared on the same line.

Own Id: OTP-19157 Aux Id: PR-8638

STDLIB 5.2.3

Fixed Bugs and Malfunctions

Fix shell expansion of -type a() :: $a. in the erlang shell.

Own Id: OTP-19062

Fix the shell Job Control Mode to not crash when typing TAB or CTRL+R.

Own Id: OTP-19072 Aux Id: PR-8391

STDLIB 5.2.2

Fixed Bugs and Malfunctions

Attempting to use the maybe construct in a macro argument could crash the compiler.

Own Id: OTP-19031 Aux Id: GH-8268

STDLIB 5.2.1

Fixed Bugs and Malfunctions

The help texts shown by argparse will now display sub-command arguments in the correct order.

Own Id: OTP-18900 Aux Id: PR-7945, GH-7934

Clarified the argparse documentation regarding the user-defined help template.

Own Id: OTP-18937

Fix shell expansion to not crash when expanding invalid using invalid atoms.

Own Id: OTP-18953 Aux Id: GH-8016 PR-8075

STDLIB 5.2

Fixed Bugs and Malfunctions

Make shell_docs correctly trim the newline at the end of code blocks.

Own Id: OTP-18777 Aux Id: PR-7663

Replaced unintentional Erlang Public License 1.1 headers in some files with the intended Apache License 2.0 header.

Own Id: OTP-18815 Aux Id: PR-7780

Fixed a bug where autocompletion could crash the shell when trying to expand a nested tuple.

Own Id: OTP-18822 Aux Id: PR-7796

Removed auto closing feature, in autocompletion, for function arguments, tuples, records and maps, since this could interfere with autocompletion of atoms.

Own Id: OTP-18823

Fixed a bug where autocompletion string formatting would remove suggestions that had the same name but different case.

Own Id: OTP-18824

Fix so that ctrl+h, ctrl+backspace in the shell only removes one character instead of a whole word.

Own Id: OTP-18826 Aux Id: PR-7797

Fix so that its possible to override the default keyboard shortcuts for the shell.

Own Id: OTP-18827 Aux Id: PR-7797

Allow shell local func v(), in a restricted shell

Own Id: OTP-18828 Aux Id: PR-7799

Report syntax error when writing an invalid attribute like '1> -hej.'

Own Id: OTP-18829 Aux Id: PR-7799

When attempting to match part of a record in the key of a map generator, the entire record would be matched.

Own Id: OTP-18866 Aux Id: GH-7875, PR-7878

Improvements and New Features

The warning for accidental use of a future triple-quoted string delimiter has been upgraded to instead warn for adjacent strings without intervening white space, which effectively is the same at a string start, but also covers the same situation at a string end.

Own Id: OTP-18821 Aux Id: OTP-18746

The removal of the deprecated slave module, originally planned for OTP 27, has been postponed to OTP 29.

Own Id: OTP-18840 Aux Id: PR-7629

Guards have been added to gen_*:start* API functions to catch bad arguments earlier. Before this change, in some cases, a bad argument could tag along and cause the server to fail later, right after start.

Own Id: OTP-18857 Aux Id: GH-7685

STDLIB 5.1.1

Improvements and New Features

Garbage collect the shell process when reducing the amount of saved history and results.

Own Id: OTP-18773 Aux Id: PR-7691

STDLIB 5.1

Fixed Bugs and Malfunctions

The compiler could run forever when compiling a call to is_record/3 with a huge positive tuple size. The call is_record(A, a, 0) would crash the compiler when used in a function body. When used in a guard the compiler would emit incorrect code that would accept {a> as a record.

Own Id: OTP-18605 Aux Id: GH-7298, GH-7317

Fix bug in ets:tab2file that could make it fail if another Erlang process created the same file at the same time.

Own Id: OTP-18614 Aux Id: GH-7162, PR-7237

An {else_clause,Value} exception will now be reported nicely in the shell.

Own Id: OTP-18616 Aux Id: GH-7258

Correct return value for error case, so that it matches the documented and intended return value {error, {already_started, pid()} when local registered names are used.

Own Id: OTP-18627 Aux Id: PR-7072

sys:get_state/1,2 and sys:replace_state/2,3 has been corrected to handle a state named error as a state name, not as a failed system callback.

For the standard server behaviours this was an issue only for gen_statem (and gen_fsm) when the state name was error, and for gen_server if the complete state was {error,_}.

Own Id: OTP-18633

Multiple problems were fixed in filelib:safe_relative_path/2. If its second argument was a path that contained symbolic links, an incorrect result patch could be returned. Also, paths were sometimes falsely considered unsafe.

Own Id: OTP-18655 Aux Id: GH-6460, PR-7208

Fix deadlock when erl.exe is used as part of a pipe on Windows and trying to set the encoding of the standard_io device.

Own Id: OTP-18675 Aux Id: PR-7473 GH-7459

Expanded the documentation about how to use the standard_io, standard_error and user I/O devices.

Added the types io:standard_io/0, io:standard:error/0 and io:user/0.

Own Id: OTP-18676 Aux Id: PR-7473 GH-7459

Fix h/2,3 to properly render multi-clause documentation.

Own Id: OTP-18683 Aux Id: PR-7502

Timers created by timer:apply_after/4, apply_interval/4, and apply_repeatedly/4 would silently fail to do the apply if it was not possible to spawn a process when the timer expired. This has now been corrected, and if the spawn fails, the system will be taken down producing a crash dump.

Own Id: OTP-18759 Aux Id: GH-7606

When an Erlang source file lacked a module definition, there would be a spurious "module name must not be empty" diagnostic for each spec in the file.

Own Id: OTP-18763 Aux Id: GH-7655

Improvements and New Features

The argument descriptions for option types in argparse have been made less ambiguous.

Own Id: OTP-18679 Aux Id: ERIERL-965

Clarified the documentation of normal shutdown reason on gen_server:call/2,3

Own Id: OTP-18690 Aux Id: PR-7511, GH-7510

Pattern matching and equivalence (=:=, =/=) comparisons on 0.0 will now raise a warning, as it will no longer be considered equivalent to -0.0 in OTP 27.

If a match on 0.0 specifically is desired (distinct from -0.0), the warning can be suppressed by writing +0.0 instead.

The arithmetic comparison operators are unaffected, including arithmetic equality (==).

* POTENTIAL INCOMPATIBILITY *

Own Id: OTP-18696

The semantics of the gen_{server,statem,event} behaviour's synchronous start behaviour introduced in OTP-26.0 with OTP-18471, has been clarified in the documentation.

Own Id: OTP-18705 Aux Id: GH-7524, OTP-18471, GH-6339, PR-6843

Added functionality to set a custom multiline prompt.

Own Id: OTP-18736 Aux Id: PR-7564

A warning for (accidental use of) Triple-Quoted Strings has been implemented as per EEP 64.

* POTENTIAL INCOMPATIBILITY *

Own Id: OTP-18746 Aux Id: PR-7313, PR-7456

The keyboard shortcuts for the shell are now configurable.

Own Id: OTP-18754 Aux Id: PR-7604 PR-7647

STDLIB 5.0.2

Fixed Bugs and Malfunctions

Fix bug where when you entered Alt+Enter in the terminal, the cursor would move to the last line, instead of moving to the next line.

Own Id: OTP-18580 Aux Id: PR-7242

Fix eof handling when reading from stdin when erlang is started using -noshell.

Own Id: OTP-18640 Aux Id: PR-7384 GH-7368 GH-7286 GH-6881

Fixed problem where output would disappear if it was received after a prompt was written in the shell.

Own Id: OTP-18652 Aux Id: PR-7242

The following functions are now much faster when given a long list or binary:

erlang:list_to_integer/1

erlang:binary_to_integer/1

erlang:binary_to_integer/2

erlang:list_to_integer/2

string:to_integer/1

Own Id: OTP-18659 Aux Id: PR-7426

STDLIB 5.0.1

Fixed Bugs and Malfunctions

The POSIX error exdev was sometimes incorrectly described as "cross domain link" in some error messages.

Own Id: OTP-18578 Aux Id: GH-7213

STDLIB 5.0

Fixed Bugs and Malfunctions

All process calls in dets have been updated to use the receive queue optimizations.

Own Id: OTP-18275 Aux Id: PR-6045

proc_lib:start*/* has become synchronous when the started process fails. This requires that a failing process use a new function proc_lib:init_fail/2,3, or exits, to indicate failure. All OTP behaviours have been fixed to do this.

All these start functions now consume the 'EXIT' message from a process link for all error returns. Previously it was only the start_link/* functions that did this, and only when the started function exited, not when it used init_ack/1,2 or init_fail/2,3 to create the return value.

* POTENTIAL INCOMPATIBILITY *

Own Id: OTP-18471 Aux Id: GH-6339, PR-6843

Fixed a bug where file:read(standard_io, ...) unexpectedly returned eof in binary mode.

Own Id: OTP-18486 Aux Id: PR-6881

In the shell, v(N) would fail to retrieve the command if the command's return value was undefined.

Own Id: OTP-18548 Aux Id: PR-6967

Improvements and New Features

The Erlang shell has been improved to support the following features:

Auto-complete variables, record names, record field names, map keys, function parameter types and filenames.

Open external editor in the shell (with C-o) to edit the current expression in an editor.

Support defining records (with types), functions and function typespecs, and custom types in the shell.

Do not save pager commands, and input to io:getline in history.

Own Id: OTP-14835 Aux Id: PR-5924

Gen_server now caches external functions for use in handle_call, handle_cast and handle_info.

Own Id: OTP-15597 Aux Id: PR-5831

The TTY/terminal subsystem has been rewritten by moving more code to Erlang from the old linked-in driver and implementing all the I/O primitives needed in a NIF instead.

On Unix platforms the user should not notice a lot of difference, besides better handling of unicode characters and fixing of some long standing bugs.

Windows users will notice that erl.exe has the same functionality as a normal Unix shell and that werl.exe has been removed and replaced with a symlink to erl.exe. This makes the Windows Erlang terminal experience identical to that of Unix.

The re-write brings with it a number of bug fixes and feature additions:

The TTY is now reset when Erlang exits, fixing zsh to not break when terminating an Erlang session.

standard_error now uses the same unicode mode as standard_io.

Hitting backspace when searching the shell history with an empty search string no longer breaks the shell.

Tab expansion now works on remote nodes started using the JCL interface.

It is now possible to configure the shell slogan and the session slogans (that is the texts that appear when you start an Erlang shell). See the kernel documentation for more details.

Added shell:start_interactive for starting the interactive shell from a non-interactive Erlang session (for example an escript).

On Windows, when starting in detached mode the standard handler are now set to nul devices instead of being unset.

Standard I/O now always defaults to unicode mode if supported. Previously the default was latin1 if the runtime system had been started with -oldshell or -noshell (for example in an escript). To send raw bytes over standard out, one now explicitly has to specify io:setopts(standard_io, [{encoding, latin1}]).

* POTENTIAL INCOMPATIBILITY *

Own Id: OTP-17932 Aux Id: PR-6144 GH-3150 GH-3390 GH-4343 GH-4225

Added the zip:zip_get_crc32/2 function to retrieve the CRC32 checksum from an opened ZIP archive.

Own Id: OTP-18159 Aux Id: PR-6904

Added the options post_process_args and detached to the peer:start function.

Own Id: OTP-18176 Aux Id: PR-6118

The re:replace/3,4 functions now accept a fun as the replacement argument.

Own Id: OTP-18221 Aux Id: PR-6197

The performance of the base64 module has been significantly improved. For example, on an x86_64 system with the JIT both encode and decode are more than three times faster than in Erlang/OTP 25.

Own Id: OTP-18228 Aux Id: GH-5639

Improved implementation of timer:apply_interval/4 reducing load on the timer server, and introduction of the new function timer:apply_repeatedly/4. timer:apply_repeatedly/4 is similar to timer:apply_interval/4, but timer:apply_repeatedly/4 prevents parallel execution of triggered apply operations which timer:apply_interval/4 does not.

Own Id: OTP-18236 Aux Id: PR-6256

The base64 module now supports encoding and decoding with an alternate URL safe alphabet, and an option for accepting or adding missing = padding characters.

Own Id: OTP-18247 Aux Id: PR-6280, PR-6711

Add shell:whereis/0 which can be used to locate the current shell process.

Own Id: OTP-18272 Aux Id: PR-6279

The Erlang shell's auto-completion when typing tab has been changed to happen after the editing current line instead of before it.

This behaviour can be configured using a the shell_expand_location STDLIB configuration parameter.

Own Id: OTP-18278 Aux Id: PR-6260

New function ets:lookup_element/4 with a Default argument returned if the key did not exist in the table. The old ets:lookup_element/3 raises a badarg exception which can be both inconvenient and slower.

Own Id: OTP-18279 Aux Id: PR-6234

Typing Ctrl+L in a shell now clears the screen and redraws the current line instead of only redrawing the current line. To only redraw the current line, you must now type Alt+L. This brings the behaviour of Ctrl+L closer to how bash and other shells work.

* POTENTIAL INCOMPATIBILITY *

Own Id: OTP-18285 Aux Id: PR-6262

peer nodes using standard_io connections now include standard error from the node in the io stream from the started node.

Own Id: OTP-18287 Aux Id: PR-5955

A limitation in the binary syntax has been removed. It is now possible to match binary patterns in parallel. Example: <<A:8>> = <<B:4,C:4>> = Bin

Own Id: OTP-18297 Aux Id: GH-6348

Improve type specification of unicode:characters_to_list().

Own Id: OTP-18301 Aux Id: PR-6350

In the lists module, the zip family of functions now takes options to allow handling lists of different lengths.

Own Id: OTP-18318 Aux Id: PR-6347

It is documented that $\^X is the ASCII code for Control X, where X is an uppercase or lowercase letter. However, this notation would work for any character X, even then it didn't make sense.

In Erlang/OTP 26, it is now documented that the following characters are also allowed to follow the \^ characters: @, [, \, ], ^, _, and ?. Attempt to use other characters will be rejected with a compiler error.

The value for $\^? is now 127 (instead of 31 as in earlier releases).

Own Id: OTP-18337 Aux Id: GH-6477, PR-6503

The binary:encode_hex/2 function has been added to allow the encoded hexadecimal digits to be in either lower or upper case.

Own Id: OTP-18354 Aux Id: PR-6297

Variants of timer:tc() with user specified time unit have been introduced.

Own Id: OTP-18355 Aux Id: PR-6507

New function math:tau/0. Returns 2*math:pi().

Own Id: OTP-18361 Aux Id: PR-6536

The BIFs min/2 and max/2 are now allowed to be used in guards and match specs.

Own Id: OTP-18367 Aux Id: GH-6544

Optimized gen_server:multi_call().

Own Id: OTP-18385 Aux Id: PR-6698

Map comprehensions as suggested in EEP 58 has now been implemented.

Own Id: OTP-18413 Aux Id: EEP-58, PR-6727

Some map operations have been optimized by changing the internal sort order of atom keys. This changes the (undocumented) order of how atom keys in small maps are printed and returned by maps:to_list/1 and maps:next/1. The new order is unpredictable and may change between different invocations of the Erlang VM.

For applications where order is important, there is a new function maps:iterator/2 for creating iterators that return the map elements in a deterministic order. There are also new modifiers k and K for the format string for io:format() to support printing map elements ordered.

Own Id: OTP-18414 Aux Id: PR-6151

Make gen_server fail "silently" with a new return value for init/1.

Own Id: OTP-18423 Aux Id: https://github.com/erlang/backlog/issues/142

Improved the selective receive optimization, which can now be enabled for references returned from other functions.

This greatly improves the performance of gen_server:send_request/3, gen_server:wait_response/2, and similar functions.

Own Id: OTP-18431 Aux Id: PR-6739

It is no longer necessary to enable a feature in the runtime system in order to load modules that are using it. It is sufficient to enable the feature in the compiler when compiling it.

That means that to use feature maybe_expr in Erlang/OTP 26, it is sufficient to enable it during compilation.

In Erlang/OTP 27, feature maybe_expr will be enabled by default, but it will be possible to disable it.

Own Id: OTP-18445

Static supervisors are very idle processes after they have started so they will now be hibernated after start to improve resource management.

Own Id: OTP-18474 Aux Id: PR-6895

Deprecates dbg:stop_clear/0 because it is simply a function alias to dbg:stop/0

Own Id: OTP-18478 Aux Id: GH-6903

Support has been added in ms_transform for the actions caller_line/0, current_stacktrace/0, and current_stacktrace/1.

Own Id: OTP-18494 Aux Id: PR-6924

The family of enumeration functions in module lists has been extended with enumerate/3 that allows a step value to be supplied.

Own Id: OTP-18495 Aux Id: PR-6943

Update Unicode to version 15.0.0.

Own Id: OTP-18500

The regular expression library powering the re module is likely to be changed in Erlang/OTP 27. See Upcoming Potential Incompatibilities.

Own Id: OTP-18511 Aux Id: PR-7017

Improved the performance of sets:subtract/2 when subtracting a small number of elements.

Own Id: OTP-18515 Aux Id: GH-6990

The linter will no longer raise warnings for underspecified opaque types.

Own Id: OTP-18518 Aux Id: GH-7015

Added the new built-in type dynamic/0 introduced in EEP-61, improving support for gradual type checkers.

Own Id: OTP-18522

The by gen_statem previously used call proxy process that was used for preventing late replies from reaching the client at timeout or connection loss has been removed. It is no longer needed since process aliases take care of this, are used, and supported by all Erlang nodes that an OTP 26 Erlang node can communicate with.

Own Id: OTP-18537 Aux Id: PR-7081

Added the argparse module for simplified argument handling in escripts and similar.

Own Id: OTP-18558 Aux Id: PR-6852

Added support for multiple line expressions and navigation in the shell. Added new keybindings:

navigate up (ctrl+up)/(alt+up)

navigate down (ctrl+down)/(alt+down)

insert newline in middle of line (alt+enter)

navigate top (alt+<)/(alt+shift+up)

navigate bottom (alt+>)/(alt+shift+down)

clear current expression (alt+c)

cancel search (alt+c)

opening editor on mac (option+o)/(alt+o)

Modifies the prompt for new lines to make it clearer that the prompt has entered multi-line mode. Supports terminal with small window size, recommend not go lower than 7 rows and 40 columns. Modifies the search prompt to support multi-line statements. Redraw the prompt after continuing from JCL menu.

Own Id: OTP-18575 Aux Id: PR-7169

STDLIB 4.3.1.6

Fixed Bugs and Malfunctions

Fixed an error in uri_string:percent_decode spec

Own Id: OTP-19380 Aux Id: GH-8755

STDLIB 4.3.1.5

Fixed Bugs and Malfunctions

With this change, shutdown procedure handles a race condition between supervisor executing a shutdown and child process termination from other reason.

Own Id: OTP-19256 Aux Id: PR-8780

With this change, uri_string:normalize assumes empty path (do not crash) when no path is provided in the URI map.

Own Id: OTP-19266 Aux Id: ERIERL-1127, PR-8890

STDLIB 4.3.1.4

Fixed Bugs and Malfunctions

Attempting to use the maybe construct in a macro argument could crash the compiler.

Own Id: OTP-19031 Aux Id: GH-8268

STDLIB 4.3.1.3

Improvements and New Features

Garbage collect the shell process when reducing the amount of saved history and results.

Own Id: OTP-18773 Aux Id: PR-7691

STDLIB 4.3.1.2

Fixed Bugs and Malfunctions

The following functions are now much faster when given a long list or binary:

erlang:list_to_integer/1

erlang:binary_to_integer/1

erlang:binary_to_integer/2

erlang:list_to_integer/2

string:to_integer/1

Own Id: OTP-18659 Aux Id: PR-7426

STDLIB 4.3.1.1

Improvements and New Features

Static supervisors are very idle processes after they have started so they will now be hibernated after start to improve resource management.

Own Id: OTP-18556

STDLIB 4.3.1

Fixed Bugs and Malfunctions

The type specs in the erl_parse module has been updated to include the maybe construct and the ! operator.

Own Id: OTP-18506 Aux Id: GH-6956

STDLIB 4.3

Fixed Bugs and Malfunctions

Fixed a bug that would cause analysis to crash.

Own Id: OTP-18372 Aux Id: GH-6580

Fixed a crash when formatting stack traces for error reports.

Own Id: OTP-18375 Aux Id: GH-6591

Instead of crashing, the list_to_integer/1 and list_to_integer/2 BIFs now raise the system_limit exception for overlong lists that can't be converted to integers. Similarly, the string:to_integer/1 BIF now returns {error,system_limit} for overlong lists.

Own Id: OTP-18475 Aux Id: PR-6897

Improvements and New Features

Removal of non-necessary undefined types added to the state's supervisor record.

Own Id: OTP-18393 Aux Id: PR-6666

STDLIB 4.2

Fixed Bugs and Malfunctions

erl_tar can now read gzip-compressed tar files that are padded. There is a new option compressed_one for file:open/2 that will read a single member from a gzip file,

Own Id: OTP-18289 Aux Id: PR-6343

A concurrent call to ets:rename could cause ets:delete_all_objects to fail halfway through with badarg.

Own Id: OTP-18292 Aux Id: PR-6366

It is not allowed to call functions from guards. The compiler failed to reject a call in a guard when done by constructing a record with a default initialization expression that called a function.

Own Id: OTP-18325 Aux Id: GH-6465, GH-6466

The compiler could crash when using a record with complex field initialization expression as a filter in a list comprehension.

Own Id: OTP-18336 Aux Id: GH-6501, PR-6502

unicode:characters_to_binary() could build unnecessarily large call stack.

Own Id: OTP-18351 Aux Id: ERIERL-885, PR-6529

Improvements and New Features

Improve error message for ets:new/2 name clash. Say "name already exists" instead of less specific "invalid options".

Own Id: OTP-18283 Aux Id: PR-6338

STDLIB 4.1.1

Fixed Bugs and Malfunctions

peer nodes failed to halt when the process supervising the control connection crashed. When an alternative control connection was used, this supervision process also quite frequently crashed when the peer node was stopped by the node that started it which caused the peer node to linger without ever halting.

Own Id: OTP-18249 Aux Id: PR-6301

STDLIB 4.1

Fixed Bugs and Malfunctions

Fixed inconsistency bugs in global due to nodeup/nodedown messages not being delivered before/after traffic over connections. Also fixed various other inconsistency bugs and deadlocks in both global_group and global.

As building blocks for these fixes, a new BIF erlang:nodes/2 has been introduced and net_kernel:monitor_nodes/2 has been extended.

The -hidden and -connect_all command line arguments did not work if multiple instances were present on the command line which has been fixed. The new kernel parameter connect_all has also been introduced in order to replace the -connect_all command line argument.

Own Id: OTP-17934 Aux Id: PR-6007

Fix the public_key:ssh* functions to be listed under the correct release in the Removed Functionality User's Guide.

Own Id: OTP-18139 Aux Id: PR-6060

The type spec for format_status/1 in gen_statem, gen_server and gen_event has been corrected to state that the return value is of the same type as the argument (instead of the same value as the argument).

Own Id: OTP-18142 Aux Id: PR-6078

If the timer server child spec was already present in kernel_sup but it was not started, the timer server would fail to start with an {error, already_present} error instead of restarting the server.

Own Id: OTP-18146 Aux Id: PR-5983

When changing callback module in gen_statem the state_enter calls flag from the old module was used in for the first event in the new module, which could confuse the new module and cause malfunction. This bug has been corrected.

With this change some sys debug message formats have been modified, which can be a problem for debug code relying on the format.

* POTENTIAL INCOMPATIBILITY *

Own Id: OTP-18239

Improvements and New Features

There is a new configure option, --enable-deterministic-build, which will apply the deterministic compiler option when building Erlang/OTP. The deterministic option has been improved to eliminate more sources of non-determinism in several applications.

Own Id: OTP-18165 Aux Id: PR-5965

The rfc339_to_system_time/1,2 functions now allows the minutes part to be omitted from the time zone.

Own Id: OTP-18166 Aux Id: PR-6108

The receive statement in gen_event has been optimized to not use selective receive (which was never needed, and could cause severe performance degradation under heavy load).

Own Id: OTP-18194 Aux Id: PR-6199

Add new API function erl_features:configurable/0

Own Id: OTP-18199 Aux Id: PR-5790

STDLIB 4.0.1

Fixed Bugs and Malfunctions

In the initial release of Erlang/OTP 25, the expression bound to the _ pseudo-field in a record initialization would always be evaluated once, even if all other fields in the record were explicitly initialized. That would break the use case of binding the expression error(...) to _ in order to get an exception if not all fields were initialized.

The behavior of binding to _ has been reverted to the pre-OTP 25 behavior, that is, to not evaluate the expression if all fields have been bound to explicit values.

Own Id: OTP-18110 Aux Id: GH-6000

STDLIB 4.0

Fixed Bugs and Malfunctions

Improve the Erlang code linter's check of unused types.

Own Id: OTP-17370 Aux Id: GH-4784

Fix race condition in proc_lib:stop/3 where the process is not stopped when the timeout given is very short.

Own Id: OTP-17480 Aux Id: GH-4853 PR-4872

Maps are now fully supported in by ms_transform.

Own Id: OTP-17518 Aux Id: GH-4915

Fix gen_server:call with the first argument as self() to throw an error instead of failing with a timeout.

The same fix has also been done for gen_statem:call/3, gen_event:sync_notify/2 and any other functionality relying on the internal gen:call/3 function.

A similar fix was also done when using io:format/2 and the current group_leader was set to the current process.

* POTENTIAL INCOMPATIBILITY *

Own Id: OTP-17544 Aux Id: PR-5008

erl_pp printed unary - and + operators with a space between the operator and the operand. This is fixed by not having any space in between.

Own Id: OTP-17566 Aux Id: PR-5095, GH-5093

Adjust uri_string:normalize behavior for URIs with undefined port (URI string with a port colon but no port value or URI map with port => undefined).

Remove redundant normalization from http_request module.

Before this change, normalize would not remove port subcomponent in such cases and could for example return "http://localhost:" URI.

* POTENTIAL INCOMPATIBILITY *

Own Id: OTP-17627

Fix reduction counting bug in re:run that caused the function to yield too frequently when doing global matches.

Own Id: OTP-17661 Aux Id: PR-5165

Fix the memory value returned from ets:info(Tid,memory) when the read_concurrency option is used.

Before this fix the memory used by the scheduler specific lock cache lines was not counted towards the total. This caused the returned memory usage to be very incorrect on systems with many schedulers for tables with man locks.

Own Id: OTP-17832 Aux Id: PR-5494

Avoid confusion by correcting the argument order in the gen_event crash log printout.

Own Id: OTP-17878

Fixed string:next_grapheme/1 to return an empty binary in the tail for binary input for the last grapheme cluster.

* POTENTIAL INCOMPATIBILITY *

Own Id: OTP-18009 Aux Id: PR-5785

Fixed type specifications of the supervisor:sup_name/0 and supervisor:sup_ref/0 types.

Own Id: OTP-18034 Aux Id: PR-4661, GH-4622

If a default record field initialization (_ = Expr) was used even though all records fields were explicitly initialized, Expr would not be evaluated. That would not be a problem, except when Expr would bind a variable subsequently used, in which case the compiler would crash.

As an example, if record #r{} is defined to have only one field a, the following code would crash the compiler:

#r{a=[],_=V=42}, V

To fix that problem, the compiler will make sure that Expr is always evaluated at least once. The compiler will now rewrite the example to essentially:

V=42, #r{a=[]}, V

Own Id: OTP-18083

Improvements and New Features

Users can now configure ETS tables with the {write_concurrency, auto} option. This option forces tables to automatically change the number of locks that are used at run-time depending on how much concurrency is detected. The {decentralized_counters, true} option is enabled by default when {write_concurrency, auto} is active.

Benchmark results comparing this option with the other ETS optimization options are available here:

https://erlang.org/bench/ets_bench_result_lock_config.html

Own Id: OTP-15991 Aux Id: PR-5208

The format_status/2 callback for gen_server, gen_statem and gen_event has been deprecated in favor of the new format_status/1 callback.

The new callback adds the possibility to limit and change many more things than the just the state, such as the last received message, the reason for terminating and more events specific to each type of behavior. See the respective modules documentation for more details.

Own Id: OTP-17351 Aux Id: GH-4673 PR-4952

The timer module has been modernized and made more efficient, which makes the timer server less susceptible to being overloaded. The timer:sleep/1 function now accepts an arbitrarily large integer.

Own Id: OTP-17481 Aux Id: PR-4811

Add lists:enumerate/[1,2].

Own Id: OTP-17523 Aux Id: PR-4928

The configuration files .erlang, .erlang.cookie and .erlang.crypt can now be located in the XDG Config Home directory.

See the documentation for each file and filename:basedir/2 for more details.

Own Id: OTP-17554 Aux Id: GH-5016 PR-5408 OTP-17821

Support native time unit in calendar functions system_time_to_rfc3339/2 and rfc3339_to_system_time.

Own Id: OTP-17592 Aux Id: ERIERL-663, PR-5243

The tagged tuple tests and fun-calls have been optimized and are now a little bit cheaper than previously.

These optimizations become possible after making sure that all boxed terms have at least one word allocated after the arity word. This has been accomplished by letting all empty tuples refer to the same empty tuple literal which also reduces memory usage for empty tuples.

Own Id: OTP-17608

The signal queue benchmark in parallel_messages_SUITE and the ETS benchmark in ets_SUITE have benchmark result visualization HTML pages with "fill-screen" buttons to make the graphs bigger. This button did not work as intended before. When pressing the button for a graph, the last graph got replaced with a bigger version and not the one over the button. This is now fixed.

Own Id: OTP-17630

The new module peer supersedes the slave module. The slave module is now deprecated and will be removed in OTP 27.

peer contains an extended and more robust API for starting erlang nodes.

Own Id: OTP-17720 Aux Id: PR-5162

This change introduces quote and unquote functions in uri_string module - a replacement for deprecated encode and decode functions from http_uri.

Own Id: OTP-17778 Aux Id: GH-5368

In order to make it easier for the user to manage multiple outstanding asynchronous call requests, new functionality utilizing request identifier collections have been introduced in erpc, gen_server, gen_statem, and gen_event.

Own Id: OTP-17784 Aux Id: PR-5792

Update to the Unicode 14.0 specification.

Own Id: OTP-17869 Aux Id: PR-5595

The following ets types have been renamed to a clearer name: tab/0 to table/0 and comp_match_spec/0 to compiled_match_spec/0.

The types table_access/0 and table_type/0 have been exported.

Own Id: OTP-17901 Aux Id: GH-4968 PR-5649

Add support for locating .asn1 files to the default search rules of filelib:find_file/1 and filelib:find_source/1.

Own Id: OTP-17908 Aux Id: GH-5655 PR-5669

Type specifications have been added to the gen_server, and the documentation has been updated to utilize this.

This surfaced a few type violations that has been corrected in global, logger_olp and rpc.

Own Id: OTP-17915 Aux Id: PR-5751, GH-2375, GH-2690

The non-local function handler for the erl_eval can now be called with either two or three arguments. When called with three arguments, the first argument is the annotation for the node in the abstract format.

All errors during evaluation will now be passed through erlang:raise/3. If the restricted shell is active and it does not let erlang:raise/3 through, evaluation errors will be printed in less clear way. See the documentation for restricted shell in shell.

* POTENTIAL INCOMPATIBILITY *

Own Id: OTP-17925 Aux Id: PR-5631

Added filelib:ensure_path/1 that ensures that all directories for the given path exists (unlike filelib:ensure_dir/1, which will not create the last segment of the path).

Own Id: OTP-17953 Aux Id: PR-5621

The functions groups_from_list/2 and groups_from_list/3 have been added to the maps module.

Own Id: OTP-17969 Aux Id: PR-5588

gen_server has been refactored to throw more readable exceptions when a callback returns bad values in the Timeout field (timeout() | 'hibernate' | {'continue,_}), and also to verify that argument in the gen_server:enter_loop/3,4,5 API function.

Own Id: OTP-17974 Aux Id: GH-5683

The functions uniq/1 and uniq/2 for removing duplicates have been added to the lists module.

Own Id: OTP-17977 Aux Id: GH-5606, PR-5766

Added support for configurable features as described in EEP-60. Features can be enabled/disabled during compilation with options (-enable-feature Feature, -disable-feature Feature and +{feature, Feature, enable|disable}) to erlc as well as with directives (-feature(Feature, enable|disable).) in the file. Similar options can be used to erl for enabling/disabling features allowed at runtime. The new maybe expression (EEP-49) is fully supported as the feature maybe_expr. The features support is documented in the reference manual.

Own Id: OTP-17988

The function filename:safe_relative_path/1, which has been deprecated since OTP 25, has been removed. Use filelib:safe_relative_path/2 instead.

* POTENTIAL INCOMPATIBILITY *

Own Id: OTP-17991

A new PRNG have been added to the rand module: mwc59 which has been developed in collaboration with Sebastiano Vigna. It is intended for applications that need really fast pseudo-random numbers, and it comes with two output value scramblers, one fast and one thorough.

Two internal functions for the exsp generator have also been exported so they can be used outside the rand plug-in framework to shave off some overhead.

The internal splitmix64 generator has also been exported which can be useful for seeding other kinds of PRNG:s than its own.

Own Id: OTP-18011

STDLIB 3.17.2.4

Fixed Bugs and Malfunctions

The following functions are now much faster when given a long list or binary:

erlang:list_to_integer/1

erlang:binary_to_integer/1

erlang:binary_to_integer/2

erlang:list_to_integer/2

string:to_integer/1

Own Id: OTP-18659 Aux Id: PR-7426

STDLIB 3.17.2.3

Improvements and New Features

Static supervisors are very idle processes after they have started so they will now be hibernated after start to improve resource management.

Own Id: OTP-18556

STDLIB 3.17.2.2

Fixed Bugs and Malfunctions

It is not allowed to call functions from guards. The compiler failed to reject a call in a guard when done by constructing a record with a default initialization expression that called a function.

Own Id: OTP-18325 Aux Id: GH-6465, GH-6466

STDLIB 3.17.2.1

Fixed Bugs and Malfunctions

When changing callback module in gen_statem the state_enter calls flag from the old module was used in for the first event in the new module, which could confuse the new module and cause malfunction. This bug has been corrected.

With this change some sys debug message formats have been modified, which can be a problem for debug code relying on the format.

* POTENTIAL INCOMPATIBILITY *

Own Id: OTP-18239

STDLIB 3.17.2

Fixed Bugs and Malfunctions

The type specifications for shell_docs:get_doc/3, shell_docs:get_callback_doc/3, and shell_docs:get_type_doc/3 incorrectly stated that the returned Metadata was an empty map.

Own Id: OTP-18081

STDLIB 3.17.1

Fixed Bugs and Malfunctions

The compilation time is no longer recorded in BEAM files. There remained several undocumented functions that attempted to retrieve compilation times. Those have now been removed.

Own Id: OTP-17962

STDLIB 3.17

Fixed Bugs and Malfunctions

Fix rendering of nbsp on terminals that do not support unicode.

Own Id: OTP-17662 Aux Id: PR-5206

Improved the erl_error printout for when re fails to compile a regular expression to also print hints about why the compilation failed.

Own Id: OTP-17750 Aux Id: PR-5366

Fixed spec for supervisor_bridge:start_link().

Own Id: OTP-17766 Aux Id: PR-5362

Added missing shutdown clauses in supervisor which could cause erroneous error reports.

Own Id: OTP-17767 Aux Id: PR-5344

Improvements and New Features

Add the no_auto_import_types to erl_lint to allow a module to define types of the same name as a predefined type.

Own Id: OTP-17744 Aux Id: PR-5292

STDLIB 3.16.1

Fixed Bugs and Malfunctions

Fixed a bug that could cause a child to become orphaned when a supervisor died between unlinking and sending the shutdown signal to this child.

There was also a possibility for erratic supervisor reports caused by a race between a supervisor shutting down a child and that child exiting by itself at the same time.

Own Id: OTP-17649 Aux Id: GH-5193, PR-5201

STDLIB 3.16

Fixed Bugs and Malfunctions

Fix io:format with ~p to no longer interpret floats as printable characters.

Own Id: OTP-17424 Aux Id: GH-4801 PR-4803

Fix specs for base64 encode/decode functions to also include 0.

Own Id: OTP-17429 Aux Id: GH-4761

The failing call io:format("~p\n") would result in a warning for line number 0 instead of the correct line and column numbers. This has been corrected, and all warnings for failing calls to io:format() has been rephrased to make it clearer exactly what the problem is.

Own Id: OTP-17430

When the options warn_missing_spec and export_all were given, there would only be warnings for missing specs for functions that had been explicitly exported using an -export attribute.

Own Id: OTP-17434 Aux Id: GH-4772

Calling c:ls/1 with an atom whose contents is the the name of a file (as opposed to a directory) would crash.

Own Id: OTP-17463 Aux Id: GH-4916

The MODULE and MODULE_STRING macros would always appear to be defined (when tested by -ifdef), even though no -module() declaration had been seen yet. Changed so that -ifdef ?MODULE. will not consider ?MODULE defined if -module() has not been previously seen.

Own Id: OTP-17505 Aux Id: GH-4995

Fix bug with rendering of missing types and callbacks in shell_docs.

Own Id: OTP-17573 Aux Id: ERL-1264 GH-4270

When the deterministic option was given to the compiler, the ?FILE macro would be expanded to full path of the source file before the first include directive and to base part of the filename after include directive.

Own Id: OTP-17581 Aux Id: PR-5141

Fixed broken win32reg:delete_key and fixed win32reg:value for default value.

Own Id: OTP-17622 Aux Id: PR-5038

Fixed error information for the call maps:get(some_key, #{}).

Own Id: OTP-17634 Aux Id: GH-5196

Improvements and New Features

Most output functions in the io module now print extra error information when provided with invalid arguments. The functions are: io:format, io:fwrite, io:put_chars, io:nl and io:write.

Own Id: OTP-17317 Aux Id: PR-4757

EEP-54 (Provide more information about errors) now includes two new return values for the format_error callback, general and reason.

Multi-line error descriptions returned from a format_error callback are now correctly indented.

The documentation for erl_error, error/3 and Errors and Error Handling in the Erlang Reference Manual have been extended.

Own Id: OTP-17454 Aux Id: PR-4764

In the documentation for the lists module, it has been clarified that predicate funs must return a boolean.

Own Id: OTP-17503 Aux Id: GH-4985

The documentation for c:c/1, c:c/2, and c:c/3 has been clarified.

Own Id: OTP-17571 Aux Id: GH-5103

STDLIB 3.15.2

Fixed Bugs and Malfunctions

Fix a bug that could cause a crash when formatting tuples using the control sequences p or P and limiting the output with the option chars_limit.

Own Id: OTP-17525 Aux Id: GH-5053

STDLIB 3.15.1

Fixed Bugs and Malfunctions

Fix a bug that could cause a loop when formatting terms using the control sequences p or P and limiting the output with the option chars_limit.

Own Id: OTP-17459 Aux Id: GH-4824, GH-4842

STDLIB 3.15

Fixed Bugs and Malfunctions

Time-outs in gen_statem with relative time 0 did not behave quite according to the intended model. This has now been corrected.

The correction introduces a small potential incompatibility e.g when combining a state time-out with inserted events, and the inserted event does a state change in the state with the time-out. Before this correction the state time-out could be delivered even after the second state change, but now it is guaranteed that a state time-out is only delivered in the state it was started for, even in this corner case.

* POTENTIAL INCOMPATIBILITY *

Own Id: OTP-15107 Aux Id: ERL-1381, PR-2813

Fix bugs in erl_eval concerning bitstring comprehensions.

Own Id: OTP-16865

File names that start with a dot (such as ".gitignore" are now treated as file names and not extensions by filename:extension/1 and filename:rootname/1.

Own Id: OTP-16905

Fixed a bug where beam_lib:chunks/3 with the allow_missing_chunks option would crash if a named chunk was missing.

Own Id: OTP-16950 Aux Id: ERL-1378

A floating point zero (0.0) can be both positive (+0.0) and negative (-0.0). Multiple bugs in the compiler, runtime system, and STDLIB have been fixed to ensure that the minus sign on 0.0 is not lost.

Own Id: OTP-17077 Aux Id: ERL-1431, PR-2903, PR-2905, PR-2906

Eliminated a Dialyzer crashed when the -MMD option is used to generate a dependency file and a BEAM file a the same time.

Own Id: OTP-17118 Aux Id: PR-2825

Fixed bug in shell_docs and erl_docgen that interpreted em tags as strong.

Own Id: OTP-17122

On Solaris, the math:acos/1 and math:asin/1 functions would not fail for arguments outside the valid domain.

Own Id: OTP-17133

Silence unused_record warnings when using ms_transform. The parse transform ms_transform replaces records with tuples, which can cause the Erlang code linter to emit warnings about unused records.

Own Id: OTP-17186

Documented a deficiency in the re module regarding the [:ascii:] character class matching Latin-1 characters.

Own Id: OTP-17222 Aux Id: GH-4544

Fixed spec of start functions in generic behaviors.

Own Id: OTP-17342 Aux Id: GH-4725 PR-4726

Supervisors rejected child specs with a shutdown value of 0.

Own Id: OTP-17364 Aux Id: PR-4747

Improvements and New Features

In the rand module it is now possible to seed the default algorithm using an algorithm alias: default.

Generating pseudo random binaries has been implemented with rand:bytes/1 and rand:bytes_s/2.

Own Id: OTP-14646 Aux Id: PR-2920

New functions have been added to the proplists module: to_map/1,2 and from_map/1.

Own Id: OTP-14647 Aux Id: PR-2910

New functions have been added to the queue module: all/2, any/2, delete/2, delete_r/2, delete_with/2, and delete_with_r/2.

Own Id: OTP-14650 Aux Id: PR-2850

New function have been added to the queue module: fold/2 and filtermap/2.

Own Id: OTP-14793 Aux Id: PR-2791

Support for handling abstract code created before OTP R15 has been dropped.

Own Id: OTP-16678 Aux Id: PR-2627

Extended error information for failing BIF calls as proposed in EEP 54 has been implemented.

When a BIF call from the Erlang shell fails, more information about which argument or arguments that were in error will be printed. The same extended error information will by proc_lib, common_test, and qlc when BIF calls fail.

For applications that wish to provide the same extended error information, there are new functions erl_error:format_exception/3 and erl_error:format_exception/4.

There is a new error/3 BIF that allows applications or libraries to provide extended error information in the same way for their own exceptions.

Own Id: OTP-16686

The process alias feature as outlined by EEP 53 has been introduced. It is introduced in order to provide a lightweight mechanism that can prevent late replies after timeout or connection loss. For more information, see EEP 53 and the documentation of the new alias/1 BIF and the new options to the monitor/3 BIF.

The call operation in the framework used by gen_server, gen_statem, and gen_event has been updated to utilize alias in order to prevent late responses. The gen_statem behavior still use a proxy process in the distributed case, since it has always prevented late replies and aliases wont work against pre OTP 24 nodes. The proxy process can be removed in OTP 26.

The alias feature also made it possible to introduce new functions similar to the erpc:receive_response() function in the gen behaviors, so the new functions gen_server:receive_response(), gen_statem:receive_response(), gen_event:receive_response() have also been introduced.

Own Id: OTP-16718 Aux Id: PR-2735

Improved documentation about exit signals emitted when a gen_server terminates.

Own Id: OTP-16910 Aux Id: PR-2771

New functions have been added to the maps module: merge_with/3, intersect/2, intersect_with/3, filtermap/2, from_keys/2, and maps:foreach/2.

maps:merge_with/3 is the same as merge/2 but takes an extra fun that is used to combine items with the same key.

maps:intersect/2 computes the intersection of two maps.

maps:intersect_with/3 is the same as intersect/2 but takes an extra fun that is used to combine intersecting items.

maps:filtermap/2 allows filtering and mapping of a map in a single pass.

maps:from_keys/2 constructs a map from a list of keys and a single value and can be used to to optimize sets operations such as from_list/1, filter/2, intersection/2, and subtract/2.

maps:foreach/2 allows iteration over a map without returning any value.

Own Id: OTP-16936 Aux Id: ERL-1367

The experimental HiPE application has been removed, together with all related functionality in other applications.

* POTENTIAL INCOMPATIBILITY *

Own Id: OTP-16963

The filename:src/1 function which was deprecated in OTP 20 has been removed. Use filelib:find_source/1,3 instead.

* POTENTIAL INCOMPATIBILITY *

Own Id: OTP-16971

The pretty printer for floating point number have been changed to make it easier to see if the integer part of the number has been rounded. After the change the digit that may have been rounded always appears last or just before the exponent character (e or E). This is accomplished by always printing the number using scientific notation if it is so large that the integer part could be rounded.

Own Id: OTP-16980 Aux Id: ERL-1308

Accept references up to a size of 160-bits from remote nodes. This is the first step in an upgrade path toward using references up to 160-bits in a future OTP release.

Own Id: OTP-17005 Aux Id: OTP-16718

Add option location to erl_parse:abstract/2.

Own Id: OTP-17024

All long running functions in the maps API are now yielding. In previous releases the functions maps:from_list/1, maps:keys/1 and maps:values/1 did not yield. This could cause unfair scheduling of processes.

Own Id: OTP-17057

The sets module now has an optional map-based implementation, as described in EEP 50.

To use this implementation, pass the {version,2} option to sets:new/1 or sets:from_list/2.

Own Id: OTP-17059 Aux Id: PR-2864

Added shell_docs:supported_tags/0. This function can be used to retrieve the tags currently supported by shell_docs.

Own Id: OTP-17120

The application/erlang+html documentation storage format used by shell_docs has been updated to include the tags b, strong, h4, h5 and h6.

Own Id: OTP-17121

Do not pretty-print catch expressions with unnecessary parentheses. The re-write of the Erlang parser grammar in PR-2584 implies that parentheses around catch expressions are in many cases no longer required.

Own Id: OTP-17169 Aux Id: PR-2584

Improved explanation of {continue,Continue} in Module:init/1 of the gen_server documentation.

Own Id: OTP-17171 Aux Id: PR-3011

The erl_eval module now accepts a map for keeping track of bindings. Using an orddict for bindings will still work.

Own Id: OTP-17175

Documented epp:scan_erl_form/1 and added epp:scan_file/2.

Own Id: OTP-17199 Aux Id: PR-2658

The standard floating point printing algorithm used by the io and io_lib modules has been changed from the algorithm described in [1] to the Ryu algorithm [2]. This gives a significant speed improvement for the printing of most floating point numbers and a small memory consumption improvement.

[1]: Robert G. Burger and R. Kent Dybvig. 1996. Printing floating-point numbers quickly and accurately. In Proceedings of the ACM SIGPLAN 1996 conference on Programming language design and implementation (PLDI '96). Association for Computing Machinery, New York, NY, USA, 108–116. DOI:https://doi.org/10.1145/231379.231397

[2]: Ulf Adams. 2018. Ryū: fast float-to-string conversion. In Proceedings of the 39th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI 2018). Association for Computing Machinery, New York, NY, USA, 270–282. DOI:https://doi.org/10.1145/3192366.3192369

Thanks to Thomas Depierre

Own Id: OTP-17210

Add hex encoding and decoding functions in the binary module.

Own Id: OTP-17236 Aux Id: PR-3014

The undocumented and partially broken ets:filter/3 function has been removed.

Own Id: OTP-17263

Add support in shell_docs to display any "text" documentation format. This means that h(Module) in the shell now can display the "text/markdown" of Elixir documentation.

Own Id: OTP-17267

The internal hashing of keys within ETS tables of types set, bag, duplicate_bag has been salted to diverge from erlang:phash2. This to avoid bad hashing if phash2 is used to distribute the keys over separate tables/nodes.

Own Id: OTP-17276 Aux Id: PR-2979

Updated to the Unicode 13.0 specification.

Own Id: OTP-17327 Aux Id: PR-4707

Add compiler option {nowarn_unused_record, RecordNames}. Document compiler option nowarn_unused_type.

Own Id: OTP-17330

Implementation of EEP 56 in supervisor. It adds the concept of significant children as well as the auto_shutdown supervisor flag.

See the supervisor manual page for more information.

Own Id: OTP-17334 Aux Id: PR-4638, EEP-56

Fixed warnings in code matching on underscore prefixed variables.

Own Id: OTP-17385 Aux Id: OTP-17123

STDLIB 3.14.2.3

Fixed Bugs and Malfunctions

It is not allowed to call functions from guards. The compiler failed to reject a call in a guard when done by constructing a record with a default initialization expression that called a function.

Own Id: OTP-18325 Aux Id: GH-6465, GH-6466

STDLIB 3.14.2.2

Fixed Bugs and Malfunctions

Fix a bug that could cause a crash when formatting tuples using the control sequences p or P and limiting the output with the option chars_limit.

Own Id: OTP-17525 Aux Id: GH-5053

STDLIB 3.14.2.1

Fixed Bugs and Malfunctions

Fix a bug that could cause a loop when formatting terms using the control sequences p or P and limiting the output with the option chars_limit.

Own Id: OTP-17459 Aux Id: GH-4824, GH-4842

STDLIB 3.14.2

Fixed Bugs and Malfunctions

Dictionaries that have become zipped by the zip module did not get executable permission (for the file owner) which makes the files inside the dictionary inaccessible. This is fixed by giving dictionaries inside a zip archive XRW permission.

Own Id: OTP-17295 Aux Id: GH-4687

STDLIB 3.14.1

Fixed Bugs and Malfunctions

Handle maps in erl_parse:tokens().

Own Id: OTP-16978

The erlang shell function rr has been fixed to be able to read records from files within a code archive.

Own Id: OTP-17182 Aux Id: PR-3002

If beam_lib is asked to return abstract code for a BEAM file produced by Elixir and Elixir is not installed on the computer, beam_lib will no longer crash, but will return an error tuple. The cover:compile_beam() and cover:compile_beam_directory() functions have been updated to also return an error tuple in that situation.

Own Id: OTP-17194 Aux Id: GH-4353

Correct example module erl_id_trans regarding the {char, C} type.

Own Id: OTP-17273

STDLIB 3.14

Fixed Bugs and Malfunctions

This change fixes the handling of deep lists in the path component when using uri_string:recompose/1.

Own Id: OTP-16941

Fix shell_docs to clear shell decorations (bold/underline) when paginating output.

Fix various small renderings issues when integrating shell_docs with edoc.

Own Id: OTP-17047

Improvements and New Features

Improved the API and documentation of the uri_string module.

Added a new chapter to the Users Guide about Uniform Resource Identifiers and their handling with the new API.

Added two new API functions: uri_string:allowed_characters/0 and uri_string:percent_decode/1.

This change has been marked as potentially incompatible as uristring:normalize/2 used to decode percent-encoded character triplets that corresponded to characters not in the reserved set. After this change, uri_string:normalize/2 will only decode those percent-encoded triplets that correspond to characters in the unreserved set (ALPHA / DIGIT / "-" / "." / "" / "~").

* POTENTIAL INCOMPATIBILITY *

Own Id: OTP-16460

The shell_docs module has been expanded with the possibility to configure unicode, ansi and column size for the rendered text.

Own Id: OTP-16990

STDLIB 3.13.2

Fixed Bugs and Malfunctions

The functions digraph:in_edges/2 and digraph:out_edges/2 would return false edges if called for a vertex that had a '_' atom in its name term.

Own Id: OTP-16655

filelib:wildcard("not-a-directory/..") should return an empty list. On Windows it returned "not-a-directory/..".

Own Id: OTP-16700

Fix the typespec of shell_docs:render to use the correct type for an MFA.

Own Id: OTP-16739

Fix uri_string:recompose/1 when host is present but input path is not absolute.

This change prevents the recompose operation to change the top level domain of the host when the path does not start with a slash.

Own Id: OTP-16751 Aux Id: ERL-1283

The epp module would return a badly formed error term when an 'if' preprocessor directive referenced an undefined symbol. epp:format_error/1 would crash when called with the bad error term.

Own Id: OTP-16816 Aux Id: ERL-1310

lists:sublist(List, Start, Len) failed with an exception if Start > length(List) + 1 even though it is explicitly documented that "It is not an error for Start+Len to exceed the length of the list".

Own Id: OTP-16830 Aux Id: ERL-1334, PR-2718

STDLIB 3.13.1

Fixed Bugs and Malfunctions

When a temporary child of a simple_one_for_one supervisor died, the internal state of the supervisor would be corrupted in a way that would cause the supervisor to retain the start arguments for subsequent children started by the supervisor, causing unnecessary growth of the supervisor's heap. There state corruption could potentially cause other problems as well.

Own Id: OTP-16804

STDLIB 3.13

Fixed Bugs and Malfunctions

Compiling a match specification with excessive nesting caused the runtime system to crash due to scheduler stack exhaustion. Instead of crashing the runtime system, effected functions will now raise a system_limit error exception in this situation.

Own Id: OTP-16431 Aux Id: ERL-592

Initialization of record fields using _ is no longer allowed if the number of affected fields is zero.

Own Id: OTP-16516

Fix bugs in eval_bits.

Own Id: OTP-16545

Improvements and New Features

Improved the printout of single line logger events for most of the OTP behaviours in STDLIB and Kernel. This includes proc_lib, gen_server, gen_event, gen_statem, gen_fsm, supervisor, supervisor_bridge and application.

Improved the chars_limit and depth handling in proc_lib and when formatting of exceptions.

Own Id: OTP-15299

Remove usage and documentation of old requests of the I/O-protocol.

Own Id: OTP-15695

Improved ETS scalability of concurrent calls that change the size of a table, like ets:insert/2 and ets:delete/2.

This performance feature was implemented for ordered_set in OTP 22.0 and does now apply for all ETS table types.

The improved scalability may come at the cost of longer latency of ets:info(T,size) and ets:info(T,memory). A new table option decentralized_counters has therefore been added. It is default true for ordered_set with write_concurrency enabled and default false for all other table types.

Own Id: OTP-15744 Aux Id: OTP-15623, PR-2229

Handle Unicode filenames in the zip module.

Own Id: OTP-16005 Aux Id: ERL-1003, ERL-1150

Unicode support was updated to the Unicode 12.1 standard.

Own Id: OTP-16073 Aux Id: PR-2339

All of the modules proc_lib, gen_server, gen_statem, and gen_event have been extended with a start_monitor() function. For more information, see the documentation of start_monitor() for these modules.

Own Id: OTP-16120 Aux Id: ERIERL-402, PR-2427

Updates for new erlang:term_to_iovec() BIF.

Own Id: OTP-16128 Aux Id: OTP-15618

Documented a quirk regarding extraction from file descriptors in erl_tar.

Own Id: OTP-16171 Aux Id: ERL-1057

Added ok as return value to gen_server:reply/2

Own Id: OTP-16210 Aux Id: PR-2411

New functions have been added to c for printing embedded documentation for Erlang modules. The functions are:

h/1,2,3 - Print the documentation for a Module:Function/Arity.

ht/1,2,3 - Print the type documentation for a Module:Type/Arity.

The embedded documentation is created when building the Erlang/OTP documentation.

Own Id: OTP-16222

Add indent and linewidth to the options of the erl_pp module's functions.

Own Id: OTP-16276 Aux Id: PR-2443

Minor updates due to the new spawn improvements made.

Own Id: OTP-16368 Aux Id: OTP-15251

The compiler will now raise a warning when inlining is used in modules that load NIFs.

Own Id: OTP-16429 Aux Id: ERL-303

Refactored the internal handling of deprecated and removed functions.

Own Id: OTP-16469

Extend erl_parse:abstract/1,2 to handle external fun expressions (fun M:F/A).

Own Id: OTP-16480

Added filelib:safe_relative_path/2 to replace filename:safe_relative_path/1, which did not safely handle symbolic links.

filename:safe_relative_path/1 has been deprecated.

Own Id: OTP-16483 Aux Id: PR-2542

The module shell_docs has been added. The module contains functions for rendering, validating and normalizing embedded documentation.

Own Id: OTP-16500

Module and function auto-completion in the shell now looks at all available modules instead of only those loaded. A module is considered available if it either is loaded already or would be loaded if called.

The auto-completion has also been expanded to work in the new h/1,2,3 function in c.

Own Id: OTP-16501 Aux Id: OTP-16494, OTP-16222, OTP-16406, OTP-16499, OTP-16500, PR-2545, ERL-708

Updated the internal pcre library to 8.44.

Own Id: OTP-16557

STDLIB 3.12.1.2

Fixed Bugs and Malfunctions

Fix a bug that could cause a crash when formatting tuples using the control sequences p or P and limiting the output with the option chars_limit.

Own Id: OTP-17525 Aux Id: GH-5053

STDLIB 3.12.1.1

Fixed Bugs and Malfunctions

Fix a bug that could cause a loop when formatting terms using the control sequences p or P and limiting the output with the option chars_limit.

Own Id: OTP-17459 Aux Id: GH-4824, GH-4842

STDLIB 3.12.1

Fixed Bugs and Malfunctions

[re:run(Subject, RE, [unicode])](re:run/3) returned nomatch instead of failing with a badarg error exception when Subject contained illegal utf8 and RE was passed as a binary. This has been corrected along with corrections of reduction counting in re:run() error cases.

Own Id: OTP-16553

STDLIB 3.12

Fixed Bugs and Malfunctions

Fix type specification for uri_string:normalize/2 that may also return error().

Own Id: OTP-16322

Improve error handling in uri_string:normalize/2. This change fixes a crash when the input URI has faulty percent-encoding.

Own Id: OTP-16351

Fix minor bugs in the Erlang pretty printer (erl_pp).

Own Id: OTP-16435

Fix the Erlang parser regarding consecutive unary operators.

Own Id: OTP-16439

Let calendar:rfc3339_to_system_time() crash when the time offset is missing.

Own Id: OTP-16514 Aux Id: ERL-1182

Improvements and New Features

Implement uri_string:resolve/{2,3} that can be used to resolve a URI reference against a base URI.

Own Id: OTP-16321

In gen_statem it is now possible to change the callback module for a running server. See gen_statem's documentation for change_callback_module, push_callback_module, and pop_callback_module.

Own Id: OTP-16477 Aux Id: PR-2531

STDLIB 3.11.2

Fixed Bugs and Malfunctions

A directory traversal vulnerability has been eliminated in erl_tar. erl_tar will now refuse to extract symlinks that points outside the targeted extraction directory and will return {error,{Path,unsafe_symlink}}. (Thanks to Eric Meadows-Jönsson for the bug report and for suggesting a fix.)

Own Id: OTP-16441

STDLIB 3.11.1

Fixed Bugs and Malfunctions

The ets:update_counter/4 core dumped when given an ordered_set with write_concurrency enabled and an invalid position. This bug has been fixed.

Own Id: OTP-16378 Aux Id: ERL-1125

STDLIB 3.11

Fixed Bugs and Malfunctions

The functions unicode:characters_to_list() and unicode:characters_to_binary() raised a badarg exception instead of returning an error tuple when passed very large invalid code points as input.

Own Id: OTP-16052

Fixed a bug in the linter where list and binary comprehensions could suppress unsafe variable errors.

Own Id: OTP-16053 Aux Id: ERL-1039

Fixed incorrect type specifications for erl_tar:open/2, create/2,3, and add/4.

Own Id: OTP-16085 Aux Id: PR-2379

Fixed erroneous type spec for binary:list_to_bin/1. Argument type was changed from iodata/0 to iolist/0.

Own Id: OTP-16132 Aux Id: ERL-1041

Fix a race in pool:pspawn_link that caused a noproc error to be thrown when using it to spawn a very short lived process.

Own Id: OTP-16211

Fixed a performance issue in ETS lookup when using the compressed option and the term contained atoms. Before this fix the decompress algorithm for atoms would unnecessarily take a global lock to validate the atom.

Own Id: OTP-16316

Improvements and New Features

Added a new compiler/linter option to disable warnings for unused types (nowarn_unused_type).

Own Id: OTP-16262 Aux Id: ERIERL-435

ETS tables have been optimized to not use any locks when running in a system with only one scheduler enabled. This can provide significant performance gains for applications that use ETS tables heavily.

Own Id: OTP-16315

STDLIB 3.10

Fixed Bugs and Malfunctions

re:run() now yields when validating utf8 in a large subject.

Own Id: OTP-15836 Aux Id: ERL-876

Upgraded the ERTS internal PCRE library from version 8.42 to version 8.43. See http://pcre.org/original/changelog.txt for information about changes made to PCRE. This library implements major parts of the re regular expressions module.

Own Id: OTP-15889

The bug with ID ERL-717 has been fixed. The functions io:columns() and io:rows() only worked correctly inside interactive erlang shells before this fix. These functions returned {error,enotsup} before this fix even if stdout and stdin were connected to a terminal when they were invoked from an escript or a program started with e.g., erl -noshell.

Own Id: OTP-15959 Aux Id: ERL-717

Fixed handling of ".." and "@" in wildcards. ".." would only work when preceded by a literal pattern such as in "a/..", not when preceded by wildcard characters such as in "*/..". The combination "@/.." was also broken, and in addition "@" in a pattern could degrade performance of the wildcard matching.

Own Id: OTP-15987 Aux Id: ERL-1029

Make sure ets:fun2ms() can handle ++/2 in the head of functions when called from the shell.

Own Id: OTP-15992 Aux Id: PR-2322

Improvements and New Features

Debugging of time-outs in gen_statem has been improved. Starting a time-out is now logged in sys:log and sys:trace. Running time-outs are visible in server crash logs, and with sys:get_status. Due to this system events {start_timer, Action, State} and {insert_timout, Event, State} have been added, which may surprise tools that rely on the format of these events.

New features: The EventContent of a running time-out can be updated with {TimeoutType, update, NewEventContent}. Running time-outs can be cancelled with {TimeoutType, cancel} which is more readable than using Time = infinity.

* POTENTIAL INCOMPATIBILITY *

Own Id: OTP-15510

re:run() now avoids validating utf8 in the subject more than once in the same call. This validation could previously be performed multiple times when the global option was passed.

Own Id: OTP-15831 Aux Id: ERL-876

ETS ordered_set tables with write_concurrency enabled has got a performance issue fixed. There were no limits for the values of internal statistics counters before this fix. This could result in that the data structure sometimes reacted slowly to a change in how many parallel processes were using it.

Own Id: OTP-15906

The ordsets:union/1 is now faster when passed a long list of ordsets.

Own Id: OTP-15927

unicode:characters_to_binary() could return very small binaries as reference counted off heap binaries. This could cause an unnecessary large memory usage and an unnecessary load on the binary allocator. Small binaries are now always returned as heap binaries.

Own Id: OTP-16002 Aux Id: ERIERL-366

Display a more meaningful error message when a bad I/O server is used in a script written in Erlang (escript).

Own Id: OTP-16006 Aux Id: ERL-992

New feature ets:info(_, binary) to get information about all reference counted binaries kept by a table. This is the same kind of debug information that process_info(_, binary) returns for a process.

Own Id: OTP-16035 Aux Id: ERIERL-366

Corrected ETS documentation about the behavior of compiled match specifications when serialized through external format.

Own Id: OTP-16038 Aux Id: PR-2366

STDLIB 3.9.2

Fixed Bugs and Malfunctions

Fix a bug that could cause a loop when formatting terms using the control sequences p or P and limiting the output with the option chars_limit.

Own Id: OTP-15875 Aux Id: ERL-967

STDLIB 3.9.1

Fixed Bugs and Malfunctions

Fix a bug that could cause a failure when formatting binaries using the control sequences p or P and limiting the output with the option chars_limit.

Own Id: OTP-15847 Aux Id: ERL-957

STDLIB 3.9

Fixed Bugs and Malfunctions

Fix a bug in string:lexemes/2.

The bug was found when optimizing the handling of deep lists of Unicode characters in the string module.

Own Id: OTP-15649

A bug has been fixed in the maps implementation that could cause a crash or memory usage to grow until the machine ran out of memory. This could happen when inserting a new key-value pair with a key K1 containing a binary B1 into a map M having a key K2 with a binary B2 if the following conditions were met:

B1 =/= B2

size(B1) >= 4294967296

size(B2) >= 4294967296

size(M) >= 32

(size(B1) rem 4294967296) == (size(B2) rem 4294967296)

the first (size(B1) rem 4294967296) bytes are the same both in B1 and B2

substituting B1 in K1 with B2 would create a term with the same value as K2

The root cause of the problem is that the maps implementation only hashed the first (X rem 4294967296) bytes of binaries so that different binaries could get the same hash value independently of the hash seed.

Own Id: OTP-15707

Since the introduction of the stack trace variable, the Erlang Pretty Printer has left out the exception class throw even when the stack trace variable cannot be left out, which is not correct Erlang code. The fix is to always include the exception class throw.

Own Id: OTP-15751

record_info/2 is a pseudo-function that requires literal arguments known at compile time. Therefore, the following usage is illegal: fun record/info/2. The compiler would crash when during compilation of that kind of code. Corrected to issue a compilation error.

Own Id: OTP-15760 Aux Id: ERL-907

Improvements and New Features

A new rand module algorithm, exro928ss (Xoroshiro928**), has been implemented. It has got a really long period and good statistical quality for all output bits, while still being only about 50% slower than the default algorithm.

The same generator is also used as a long period counter in a new crypto plugin for the rand module, algorithm crypto_aes. This plugin uses AES-256 to scramble the counter which buries any detectable statistical artifacts. Scrambling is done in chunks which are cached to get good amortized speed (about half of the default algorithm).

Own Id: OTP-14461 Aux Id: PR-1857

Types related to server naming and starting have been exported from gen_statem. These are: server_name/0, server_ref/0, start_opt/0, start_ret/0 and enter_loop_opt/0.

Own Id: OTP-14724 Aux Id: PR-2056

The default algorithm for the rand module has been changed to exsss (Xorshift116**) which is a combination of the Xorshift116 (exsp) state update and a new scrambler "StarStar" from the 2018 paper "Scrambled Linear Pseudorandom Number Generators" by David Blackman and Sebastiano Vigna. This combination should not have the caveat of weak low bits that the previous default algorithm(s) have had, with the cost of about 10% lower speed. See GitHub pull request #1969.

Own Id: OTP-14731 Aux Id: PR-1969

The generic state machine behaviour gen_statem has gotten code cleanup and documentation improvements from GitHub Pull Request #1855, even though the PR itself was rejected.

Own Id: OTP-14737 Aux Id: PR-1855

Update Unicode specification to version 11.0.

Own Id: OTP-15111

ETS option write_concurrency now also affects and improves the scalability of ordered_set tables. The implementation is based on a data structure called contention adapting search tree, where the lock granularity adapts to the actual amount of concurrency exploited by the applications in runtime.

Own Id: OTP-15128

Optimized maps:new/0 with trivial Erlang implementation, making use of literal terms (the empty map) not needing dynamic heap allocation.

Own Id: OTP-15200 Aux Id: PR-1878

The gen_* behaviours have been changed so that if logging of the last N messages through sys:log/2,3 is active for the server, this log is included in the terminate report.

To accomplish this the format of "System Events" as defined in the man page for sys has been clarified and cleaned up, a new function sys:get_log/1 has been added, and sys:get_debug/3 has been deprecated. Due to these changes, code that relies on the internal badly documented format of "System Events", need to be corrected.

* POTENTIAL INCOMPATIBILITY *

Own Id: OTP-15381

The gen_statem behaviour engine loop has been optimized for better performance in particular when the callback module returns some actions, that is better performance for more realistic applications than the Echo Benchmark.

Own Id: OTP-15452

Do not allow function specifications for functions residing in other modules.

* POTENTIAL INCOMPATIBILITY *

Own Id: OTP-15563 Aux Id: ERL-845, OTP-15562

The persistent_term functions put/2 and erase/1 are now yielding.

Own Id: OTP-15615

Previously, all ETS tables used centralized counter variables to keep track of the number of items stored and the amount of memory consumed. These counters can cause scalability problems (especially on big NUMA systems). This change adds an implementation of a decentralized counter and modifies the implementation of ETS so that ETS tables of type ordered_set with write_concurrency enabled use the decentralized counter. Experiments indicate that this change substantially improves the scalability of ETS ordered_set tables with write_concurrency enabled in scenarios with frequent ets:insert/2 and ets:delete/2 calls.

Own Id: OTP-15623 Aux Id: PR-2190

Use ssh instead of rsh as the default remote shell.

Own Id: OTP-15633 Aux Id: PR-1787

Added beam_lib:strip/2 and friends, which accept a list of chunks that should be preserved when stripping.

Own Id: OTP-15680 Aux Id: PR-2114

Optimize printing of maps with io_lib:write(). Also optimize pretty printing of strings (~s and ~ts) when limiting the output with the chars_limit option.

Own Id: OTP-15705

There are new compiler options nowarn_removed and {nowarn_removed,Items} to suppress warnings for functions and modules that have been removed from OTP.

Own Id: OTP-15749 Aux Id: ERL-904

Let the Erlang Pretty Printer put atomic parts on the same line.

Own Id: OTP-15755

Add option quote_singleton_atom_types to the Erlang Pretty Printer's functions. Setting the option to true adds quotes to all singleton atom types.

Own Id: OTP-15756

STDLIB 3.8.2.4

Fixed Bugs and Malfunctions

[re:run(Subject, RE, [unicode])](re:run/3) returned nomatch instead of failing with a badarg error exception when Subject contained illegal utf8 and RE was passed as a binary. This has been corrected along with corrections of reduction counting in re:run() error cases.

Own Id: OTP-16553

STDLIB 3.8.2.3

Fixed Bugs and Malfunctions

A directory traversal vulnerability has been eliminated in erl_tar. erl_tar will now refuse to extract symlinks that points outside the targeted extraction directory and will return {error,{Path,unsafe_symlink}}. (Thanks to Eric Meadows-Jönsson for the bug report and for suggesting a fix.)

Own Id: OTP-16441

STDLIB 3.8.2.2

Fixed Bugs and Malfunctions

Fix a bug that could cause a loop when formatting terms using the control sequences p or P and limiting the output with the option chars_limit.

Own Id: OTP-15875 Aux Id: ERL-967

STDLIB 3.8.2.1

Fixed Bugs and Malfunctions

Fix a bug that could cause a failure when formatting binaries using the control sequences p or P and limiting the output with the option chars_limit.

Own Id: OTP-15847 Aux Id: ERL-957

STDLIB 3.8.2

Fixed Bugs and Malfunctions

A bug in gen_statem has been fixed where the internal timeout message could arrive as an info to the callback module during high load due to incorrect use of asynchronous timer cancel.

Own Id: OTP-15295

STDLIB 3.8.1

Fixed Bugs and Malfunctions

Fixed a performance regression when reading files opened with the compressed flag.

Own Id: OTP-15706 Aux Id: ERIERL-336

STDLIB 3.8

Fixed Bugs and Malfunctions

Fix a bug in the Erlang Pretty Printer: long atom names in combination with <<>> could cause a crash.

Own Id: OTP-15592 Aux Id: ERL-818

Fix bugs that could cause wrong results or bad performance when formatting lists of characters using the control sequences p or P and limiting the output with the option chars_limit.

Own Id: OTP-15639

Improvements and New Features

Improved ETS documentation about safe table traversal and the partially bound key optimization for ordered_set.

Own Id: OTP-15545 Aux Id: PR-2103, PR-2139

Optimize calendar:gregorian_days_to_date/1.

Own Id: OTP-15572 Aux Id: PR-2121

Optimize functions calendar:rfc3339_to_system_time() and calendar:system_time_to_rfc3339().

Own Id: OTP-15630

STDLIB 3.7.1

Fixed Bugs and Malfunctions

Optimize pretty printing of terms. The slower behaviour was introduced in Erlang/OTP 20.

Own Id: OTP-15573 Aux Id: ERIERL-306

STDLIB 3.7

Fixed Bugs and Malfunctions

Document bit_size in match specifications and allow it in ets:fun2ms.

Own Id: OTP-15343 Aux Id: PR-1962

The beam() type in beam_lib is defined as module() | file:filename() | binary(). The module/0 is misleading. Giving the module name as an atom will only work if the BEAM file is in a current directory.

To avoid confusion, module/0 has been removed from the type. That means that there will be a Dialyzer warning for code that call beam_lib with an atom as filename, but the calls will still work.

Own Id: OTP-15378 Aux Id: ERL-696

unicode_util crashed on certain emoji grapheme clusters in binary strings.

Own Id: OTP-15428 Aux Id: ERL-777

When an external fun was used, warnings for unused variables could be suppressed.

Own Id: OTP-15437 Aux Id: ERL-762

Fix reduction count in lists:member/2

Own Id: OTP-15474 Aux Id: ERIERL-229

Improvements and New Features

When specified, the +{source,Name} option will now override the actual file name in stack traces, instead of only affecting the return value of Mod:module_info().

The +deterministic flag will also affect stack traces now, omitting all path information except the file name, fixing a long-standing issue where deterministic builds required deterministic paths.

Own Id: OTP-15245 Aux Id: ERL-706

List subtraction (The -- operator) will now yield properly on large inputs.

Own Id: OTP-15371

calendar:system_time_to_rfc3339/1,2 no longer remove trailing zeros from fractions.

Own Id: OTP-15464

STDLIB 3.6

Fixed Bugs and Malfunctions

The specs of filename:basedir/2,3 are corrected.

Own Id: OTP-15252 Aux Id: ERL-667

Improvements and New Features

Let dets:open_file() exit with a badarg message if given a raw file name (a binary).

Own Id: OTP-15253 Aux Id: OTP-13229, ERL-55

The Format argument of the formatting functions in modules io and io_lib is accepted even if it is, for example, a list of binaries. This is how it used to be before Erlang/OTP 21.0.

Own Id: OTP-15304

STDLIB 3.5.1

Fixed Bugs and Malfunctions

Fix a bug that could cause a crash when formatting a list of non-characters using the control sequences p or P and limiting the output with the option chars_limit.

Own Id: OTP-15159

STDLIB 3.5

Fixed Bugs and Malfunctions

gen_statem improvements.

When using an exception that is valid but not allowed in a state enter call, the reason has been changed from {bad_action_from_state_function,Action} to {bad_state_enter_action_from_state_function,Action}.

Timer parsing has been improved. Many erroneous timeout tuples was not handled correctly.

The documentation has been updated, in particular the User's Guide and the pointer to it from the Reference Manual is much more obvious.

Own Id: OTP-14015

The type specifications for file:posix/0 and inet:posix/0 have been updated according to which errors file and socket operations should be able to return.

Own Id: OTP-14019 Aux Id: ERL-550

File operations used to accept filenames containing null characters (integer value zero). This caused the name to be truncated and in some cases arguments to primitive operations to be mixed up. Filenames containing null characters inside the filename are now rejected and will cause primitive file operations to fail.

Also environment variable operations used to accept names and values of environment variables containing null characters (integer value zero). This caused operations to silently produce erroneous results. Environment variable names and values containing null characters inside the name or value are now rejected and will cause environment variable operations to fail.

Primitive environment variable operations also used to accept the $= character in environment variable names causing various problems. $= characters in environment variable names are now also rejected.

Also os:cmd/1 now reject null characters inside its command.

erlang:open_port/2 will also reject null characters inside the port name from now on.

* POTENTIAL INCOMPATIBILITY *

Own Id: OTP-14543 Aux Id: ERL-370

Make io_lib:unscan_format/1 work with pad char and default precision.

Own Id: OTP-14958 Aux Id: PR-1735

The control sequence modifiers t and l can be used together in the same control sequence which makes it possible to have Unicode atoms and no detection of printable character lists at the same time.

Own Id: OTP-14971 Aux Id: PR-1743

Fix a bug in the Erlang code linter: the check of guard expressions no longer returns false if the map syntax is used. The bug affected the Erlang shell, the Debugger, and other modules evaluating abstract code.

Own Id: OTP-15035 Aux Id: ERL-613

A sys debug fun of type {Fun,State} should not be possible to install twice. This was, however, possible if the current State was 'undefined', which was mistaken for non-existing fun. This has been corrected.

Own Id: OTP-15049

Fix io:putchars/2 stacktrace rewriting at errors to point to a valid function.

Own Id: OTP-15101

Improvements and New Features

The gen_server has gotten a new callback handle_continue/2 for check pointing the state. This is useful at least when implementing behaviours on top of gen_server and for some start up scenarios.

Own Id: OTP-13019 Aux Id: PR-1490

The semantics of timeout parameter {clean_timeout,infinity} to gen_statem:call/3 has been changed to use a proxy process for the call. With this change clean_timeout implicates a proxy process with no exceptions. This may be a hard to observe incompatibility: in the presence of network problems a late reply could arrive in the caller's message queue when catching errors. That will not happen after this correction.

The semantics of timeout parameter infinity has not been changed.

* POTENTIAL INCOMPATIBILITY *

Own Id: OTP-13073 Aux Id: PR-1595

A new logging API is added to Erlang/OTP, see the logger manual page, and section Logging in the Kernel User's Guide.

Calls to error_logger are automatically redirected to the new API, and legacy error logger event handlers can still be used. It is, however, recommended to use the Logger API directly when writing new code.

Notice the following potential incompatibilities:

Kernel configuration parameters error_logger still works, but is overruled if the default handler's output destination is configured with Kernel configuration parameter logger.

In general, parameters for configuring error logger are overwritten by new parameters for configuring Logger.

The concept of SASL error logging is deprecated, meaning that by default the SASL application does not affect which log events are logged.

By default, supervisor reports and crash reports are logged by the default Logger handler started by Kernel, and end up at the same destination (terminal or file) as other standard log event from Erlang/OTP.

Progress reports are not logged by default, but can be enabled by setting the primary log level to info, for example with the Kernel configuration parameter logger_level.

To obtain backwards compatibility with the SASL error logging functionality from earlier releases, set Kernel configuration parameter logger_sasl_compatible to true. This prevents the default Logger handler from logging any supervisor-, crash-, or progress reports. Instead, SASL adds a separate Logger handler during application start, which takes care of these log events. The SASL configuration parameters sasl_error_logger and sasl_errlog_type specify the destination (terminal or file) and severity level to log for these events.

Since Logger is new in Erlang/OTP 21.0, we do reserve the right to introduce changes to the Logger API and functionality in patches following this release. These changes might or might not be backwards compatible with the initial version.

* POTENTIAL INCOMPATIBILITY *

Own Id: OTP-13295

Add functions calendar:system_time_to_local_time/2 and calendar:system_time_to_universal_time/2.

Own Id: OTP-13413

Functions rand:uniform_real/0 and rand:uniform_real_s/1 have been added. They produce uniformly distributed numbers in the range 0.0 =< X < 1.0 that are as close to random real numbers as Normalized IEEE 754 Double Precision allows. Because the random real number exactly 0.0 is infinitely improbable they will never return exactly 0.0.

These properties are useful when you need to call for example math:log(X) or 1 / X on a random value X, since that will never fail with a number from these new functions.

Own Id: OTP-13764 Aux Id: PR-1574

Added maps:iterator/0 and maps:next/1 to be used for iterating over the key-value associations in a map.

Own Id: OTP-14012

Changed the default behaviour of .erlang loading: .erlang is no longer loaded from the current directory. c:erlangrc(PathList) can be used to search and load an .erlang file from user specified directories.

escript, erlc, dialyzer and typer no longer load an .erlang at all.

* POTENTIAL INCOMPATIBILITY *

Own Id: OTP-14439

Added new uri_string module to stdlib for handling URIs (RFC 3986).

Own Id: OTP-14496

Update Unicode specification to version 10.0.

Own Id: OTP-14503

filelib:wildcard() now allows characters with a special meaning to be escaped using backslashes.

This is an incompatible change, but note that the use of backslashes in wildcards would already work differently on Windows and Unix. Existing calls to filelib:wildcard() needs to be updated. On Windows, directory separators must always be written as a slash.

* POTENTIAL INCOMPATIBILITY *

Own Id: OTP-14577

The supervisor now stores its child specifications in a map instead of a list. This causes a significant improvement when starting many children under a non-simple_one_for_one supervisor.

Own Id: OTP-14586

The base64 module is optimized.

Note that the functions encode/1, decode/1, and mime_decode/1 fail unless called with an argument of the documented type. They used to accept any iodata/0.

Own Id: OTP-14624 Aux Id: PR-1565

Add function lists:search/2.

Own Id: OTP-14675 Aux Id: PR-102

uri_string module extended with functions for handling application/x-www-form-urlencoded query strings based on the HTML5 specification.

Own Id: OTP-14747

Add functions calendar:rfc3339_to_system_time/1,2 and calendar:system_time_to_rfc3339/1,2.

Own Id: OTP-14764

The stack traces returned by the functions of the erl_eval module more accurately reflect where the exception occurred.

Own Id: OTP-14826 Aux Id: PR 1540

Add options atime, mtime, ctime, uid, and gid to the erl_tar:add/3,4 functions.

Own Id: OTP-14834 Aux Id: PR 1608

Added ets:whereis/1 for retrieving the table identifier of a named table.

Own Id: OTP-14884

Improved URI normalization functions in the uri_string module.

Own Id: OTP-14910

The new functions io_lib:fwrite/3 and io_lib:format/3 take a third argument, an option list. The only option is chars_limit, which is used for limiting the number of returned characters. The limit is soft, which means that the number of returned characters exceeds the limit with at most a smallish amount. If the limit is set, the functions format/3 and fwrite/3 try to distribute the number of characters evenly over the control sequences pPswW. Furthermore, the control sequences pPwP try to distribute the number of characters evenly over substructures.

A modification of the control sequences pPwW is that even if there is no limit on the number of returned characters, all associations of a map are printed to the same depth. The aim is to give a more consistent output as the order of map keys is not defined. As before, if the depth is less than the number of associations of a map, the selection of associations to print is arbitrary.

Own Id: OTP-14983

Add functions ordsets:is_empty/1 and sets:is_empty/1.

Own Id: OTP-14996 Aux Id: ERL-557, PR-1703

Improve performance of string:uppercase/1, string:lowercase/1 and string:casefold/1 when handling ASCII characters.

Own Id: OTP-14998

External funs with literal values for module, name, and arity (e.g. erlang:abs/1) are now treated as literals. That means more efficient code that produces less garbage on the heap.

Own Id: OTP-15003

sys:statistics(Pid,get) did not report 'out' messages from gen_server. This is now corrected.

Own Id: OTP-15047

A sys debug function can now have the format {Id,Fun,State} in addition to the old {Fun,State}. This allows installing multiple instances of a debug fun.

Own Id: OTP-15048

The lib module is removed:

lib:error_message/2 is removed.

lib:flush_receive/0 is removed.

lib:nonl/1 is removed.

lib:progname/0 is replaced by ct:get_progname/0.

lib:send/2 is removed.

lib:sendw/2 is removed.

* POTENTIAL INCOMPATIBILITY *

Own Id: OTP-15072 Aux Id: PR 1786, OTP-15114

Function ets:delete_all_objects/1 now yields the scheduler thread for large tables that take significant time to clear. This to improve real time characteristics of other runnable processes.

Own Id: OTP-15078

In control sequences of the functions io:fwrite/2,3 and io_lib:fwrite/2,3 containing p or P, a field width of value 0 means that no line breaks are inserted. This is in contrast to the old behaviour, where 0 used to insert line breaks after every subterm. To insert line breaks after every subterm, a field width of value 1 can be used.

* POTENTIAL INCOMPATIBILITY *

Own Id: OTP-15103 Aux Id: ERL-607

STDLIB 3.4.5.1

Improvements and New Features

List subtraction (The -- operator) will now yield properly on large inputs.

Own Id: OTP-15371

STDLIB 3.4.5

Fixed Bugs and Malfunctions

The Module:init/1 function in gen_statem may return an actions list containing any action, but an erroneous check only allowed state enter actions so e.g {next_event,internal,event} caused a server crash. This bug has been fixed.

Own Id: OTP-13995

STDLIB 3.4.4

Fixed Bugs and Malfunctions

Correct filelib:find_source() and filelib:find_file() to by default also search one level below src. This is in accordance with the Design Principles which states that an application can have Erlang source files one level below the src directory.

Own Id: OTP-14832 Aux Id: ERL-527

The contract of erl_tar:table/2 is corrected.

Own Id: OTP-14860 Aux Id: PR 1670

Correct a few contracts.

Own Id: OTP-14889

Fix string:prefix/2 to handle an empty string as second argument.

Own Id: OTP-14942 Aux Id: PR-1702

STDLIB 3.4.3

Fixed Bugs and Malfunctions

Make ets:i/1 exit cleaner when ^D is input while browsing a table. Only the old Erlang shell is affected (erl flag -oldshell).

Own Id: OTP-14663

Fixed handling of windows UNC paths in module filename.

Own Id: OTP-14693

Improvements and New Features

Improve performance of the new string functionality when handling ASCII characters.

Own Id: OTP-14670

Added a clarification to the documentation of unicode:characters_to_list/2.

Own Id: OTP-14798

STDLIB 3.4.2

Fixed Bugs and Malfunctions

Fix a bug in the Erlang shell where recursively defined records with typed fields could cause a loop.

Own Id: OTP-14488 Aux Id: PR-1489

Make edlin handle grapheme clusters instead of codepoints to improve the handling multi-codepoints characters.

Own Id: OTP-14542

There could be false warnings for erlang:get_stacktrace/0 being used outside of a try block when using multiple catch clauses.

Own Id: OTP-14600 Aux Id: ERL-478

Improvements and New Features

The Erlang code linter no longer checks that the functions mentioned in nowarn_deprecated_function options are declared in the module.

Own Id: OTP-14378

General Unicode improvements.

Own Id: OTP-14462

STDLIB 3.4.1

Fixed Bugs and Malfunctions

A bug in proc_lib:format() introduced in Erlang/OTP 20.0 is corrected.

Own Id: OTP-14482 Aux Id: PR-1488

Fix string:len/1 to be compatible with previous versions.

Own Id: OTP-14487 Aux Id: ERIERL-40

In OTP-20.0, the behavior of c, make, and ct_make was changed so that in some cases the beam files by default would be written to the directory where the source files were found. This is now changed back to the old behavior so beam files are by default written to current directory.

Own Id: OTP-14489 Aux Id: ERL-438

STDLIB 3.4

Fixed Bugs and Malfunctions

For many releases, it has been legal to override a BIF with a local function having the same name. However, calling a local function with the same name as guard BIF as filter in a list comprehension was not allowed.

Own Id: OTP-13690

A new (default) pseudo-random number generator algorithm Xoroshiro116+ has been implemented in the rand module.

The old algorithm implementations had a number of flaws so they are all deprecated, but corrected versions of two of them have been added. See the documentation.

Own Id: OTP-14295 Aux Id: PR-1372

The Erlang shell, qlc:string_to_handle(), and the Debugger (the Evaluator area and Edit variable window of the Bindings area) can parse pids, ports, references, and external funs, as long as they can be created in the running system.

Own Id: OTP-14296

Internal code change: Calls to catch followed by a call to erlang:get_stacktrace/0 has been rewritten to use try instead of catch to make the code future-proof.

Own Id: OTP-14400

The ms_transform module, used by ets:fun2ms/1 and dbg:fun2ms/1, evaluates constant arithmetic expressions. This is necessary since the Erlang compiler, which normally evaluates constant expressions, does not recognize the format generated by ms_transform.

Own Id: OTP-14454 Aux Id: ERIERL-29

The state machine engine gen_statem can now handle generic time-outs (multiple named) as well as absolute time-out time. See the documentation.

The gen_statem callback Module:init/1 has become mandatory to harmonize with other gen_* modules. This may be an incompatibility for gen_statem callback modules that use gen_statem:enter_loop/4-6.

* POTENTIAL INCOMPATIBILITY *

Own Id: OTP-14531

Improvements and New Features

Improved unicode support for strings. Added normalization functions in the unicode module. Extended the string module API with new functions with improved unicode handling and that works on grapheme clusters. The new functions operates on the unicode:chardata() type, thus they also accept UTF-8 binaries as input.

The old string API have been marked as obsolete. The return values have been changed for some error cases.

* POTENTIAL INCOMPATIBILITY *

Own Id: OTP-10289 Aux Id: OTP-10309

There are two new guard BIFs 'floor/1' and 'ceil/1'. They both return integers. In the 'math' module, there are two new BIFs with the same names that return floating point values.

Own Id: OTP-13692

Making code_change, terminate and handle_info callbacks optional in the OTP behaviours.

Own Id: OTP-13801

The support for Dets files created with Erlang/OTP R7 and earlier is removed.

Own Id: OTP-13830

Replaced usage of deprecated symbolic time unit representations.

Own Id: OTP-13831 Aux Id: OTP-13735

The function fmod/2 has been added to the math module.

Own Id: OTP-14000

The EXIT signals received from processes using proc_lib now looks like EXIT signals from processes that were spawned using spawn_link. In particular, that means that the stack trace is now included in the EXIT signal so that it can see where the process crashed.

Own Id: OTP-14001

sets:add_element/2 is faster when adding an element that is already present, and sets:del_element/2 is faster when the element to be deleted is not present. This optimization can make certain operations, such as sets:union/2 with many overlapping elements, up to two orders of magnitude faster.

Own Id: OTP-14035

Add information in doc about supervisor shutdown reason when maximum restart frequency is reached.

Own Id: OTP-14037 Aux Id: PR-1233

Added rand:jump/[0|1] functions.

Own Id: OTP-14038 Aux Id: PR-1235

Functions for detecting changed code has been added. code:modified_modules/0 returns all currently loaded modules that have changed on disk. code:module_status/1 returns the status for a module. In the shell and in c module, mm/0 is short for code:modified_modules/0, and lm/0 reloads all currently loaded modules that have changed on disk.

Own Id: OTP-14059

Each assert macro in assert.hrl now has a corresponding version with an extra argument, for adding comments to assertions. These can for example be printed as part of error reports, to clarify the meaning of the check that failed.

Own Id: OTP-14066

error_logger_tty_h and error_logger_file_h now inserts the node information for nonlocal messages before the message itself instead of after, both for readability and so as not to change the line termination property at the end of the message.

Own Id: OTP-14068

The Erlang code linter checks for badly formed type constraints.

Own Id: OTP-14070 Aux Id: PR-1214

By default, there will now be a warning when export_all is used. The warning can be disabled using nowarn_export_all.

Own Id: OTP-14071

When a gen_server process crashes, the stacktrace for the client will be printed to facilitate debugging.

Own Id: OTP-14089

Optimized ETS operations by changing table identifier type from integer to reference. The reference enables a more direct mapping to the table with less potential lock contention and makes especially creation and deletion of tables scale much better.

The change of the opaque type for the ETS table identifiers may cause failure in code that make faulty assumptions about this opaque type.

Note

The number of tables stored at one Erlang node used to be limited. This is no longer the case (except by memory usage). The previous default limit was about 1400 tables and could be increased by setting the environment variable ERL_MAX_ETS_TABLES before starting the Erlang runtime system. This hard limit has been removed, but it is currently useful to set the ERL_MAX_ETS_TABLES anyway. It should be set to an approximate of the maximum amount of tables used. This since an internal table for named tables is sized using this value. If large amounts of named tables are used and ERL_MAX_ETS_TABLES hasn't been increased, the performance of named table lookup will degrade.

* POTENTIAL INCOMPATIBILITY *

Own Id: OTP-14094

take/2 has been added to dict, orddict, and gb_trees. take_any/2 has been added to gb_trees.

Own Id: OTP-14102

Extend gen_event API to handle options as well.

Own Id: OTP-14123

Advice on how to tune the supervisor restart frequency (intensity and period) is added to System Documentation - Design Principles - Supervisor Behaviour.

Own Id: OTP-14168 Aux Id: PR-1289

gen_fsm is deprecated and is replaced by gen_statem, however for backwards compatibility reasons gen_fsm may continue to exist as an undocumented feature for quite some time.

Own Id: OTP-14183

The shell functions c/1 and c/2 have been extended so that if the argument is a module name instead of a file name, it automatically locates the .beam file and the corresponding source file, and then recompiles the module using the same compiler options (plus any options passed to c/2). If compilation fails, the old beam file is preserved. Also adds c(Mod, Opts, Filter), where the Filter argument allows you to remove old compiler options before the new options are added.

New utility functions file_find/2/3 and find_source/1/2/3 have been added to filelib.

Own Id: OTP-14190

erl_tar in previous versions of OTP only supports the USTAR format. That limited path names to at most 255 bytes, and did not support Unicode characters in names in a portable way.

erl_tar now has support for reading tar archives in the formats currently in common use, such as v7, STAR, USTAR, PAX, and GNU tar's extensions to the STAR/USTAR format. When writing tar archives, erl_tar can now write them in the PAX format if necessary (for example, to support very long filenames or filenames with Unicode characters). If possible, erl_tar will still write tar archives in the USTAR for maximum portability.

Own Id: OTP-14226

base64:mime_decode/1 has been optimized so that it is now almost as fast asbase64:decode/1; it used be noticeably slower.

Own Id: OTP-14245

erl_tar will now strip any leading '/' from pathnames when extracting files from a tar archive and write a message to the error logger. There is also new check for directory traversal attacks; if a relative path points above the current working directory the extraction will be aborted.

Own Id: OTP-14278

Miscellaneous updates due to atoms containing arbitrary Unicode characters.

Own Id: OTP-14285

The Crypto application now supports generation of cryptographically strong random numbers (floats < 1.0 and integer arbitrary ranges) as a plugin to the 'rand' module.

Own Id: OTP-14317 Aux Id: PR-1372

Add new function ets:select_replace/2 which performs atomic "compare-and-swap" operations for ETS objects using match specifications.

Own Id: OTP-14319 Aux Id: PR-1076

The Erlang code linter checks for bad dialyzer attributes. It also checks for bad type variables in type declarations.

Own Id: OTP-14323

Two new functions has been implemented in the rand module; normal/2 and normal_s/3, that both produce normal distribution (pseudo) random numbers with mean value and variance according to arguments.

Own Id: OTP-14328 Aux Id: PR-1382

Upgraded the OTP internal PCRE library from version 8.33 to version 8.40. This library is used for implementation of the re regular expressions module.

Besides various bug fixes, the new version allows for better stack protection. In order to utilize this feature, the stack size of normal scheduler threads is now by default set to 128 kilo words on all platforms. The stack size of normal scheduler threads can be set upon system start by passing the +sss command line argument to the erl command.

See http://pcre.org/original/changelog.txt for information about changes made to PCRE between the versions 8.33 and 8.40.

* POTENTIAL INCOMPATIBILITY *

Own Id: OTP-14331 Aux Id: ERL-208

Added function re:version/0 which returns information about the OTP internal PCRE version used for implementation of the re module.

Own Id: OTP-14347 Aux Id: PR-1412

The format of debug information that is stored in BEAM files (when debug_info is used) has been changed. The purpose of the change is to better support other BEAM-based languages such as Elixir or LFE.

All tools included in OTP (dialyzer, debugger, cover, and so on) will handle both the new format and the previous format. Tools that retrieve the debug information using beam_lib:chunk(Beam, [abstract_code]) will continue to work with both the new and old format. Tools that call beam_lib:chunk(Beam, ["Abst"]) will not work with the new format.

For more information, see the description of debug_info in the documentation for beam_lib and the description of the {debug_info,{Backend,Data}} option in the documentation for compile.

Own Id: OTP-14369 Aux Id: PR-1367

Add option hibernate_after to gen_server, gen_statem and gen_event. Also added to the deprecated gen_fsm behaviour.

Own Id: OTP-14405

The size of crash reports created by gen_server, gen_statem and proc_lib is limited with aid of the Kernel application variable error_logger_format_depth. The purpose is to limit the size of the messages sent to the error_logger process when processes with huge message queues or states crash.

The crash report generated by proc_lib includes the new tag message_queue_len. The neighbour report also includes the new tag current_stacktrace. Finally, the neighbour report no longer includes the tags messages and dictionary.

The new function error_logger:get_format_depth/0 can be used to retrieve the value of the Kernel application variable error_logger_format_depth.

Own Id: OTP-14417

STDLIB 3.3

Fixed Bugs and Malfunctions

An escript with only two lines would not work.

Own Id: OTP-14098

Characters ($char) can be used in constant pattern expressions. They can also be used in types and contracts.

Own Id: OTP-14103 Aux Id: ERL-313

The signatures of erl_parse:anno_to_term/1 and erl_parse:anno_from_term/1 are corrected. Using these functions no longer results in false Dialyzer warnings.

Own Id: OTP-14131

Pretty-printing of maps is improved.

Own Id: OTP-14175 Aux Id: seq13277

If any of the following functions in the zip module crashed, a file would be left open: extract(), unzip(), create(), or zip(). This has been corrected.

A zip file having a "Unix header" could not be unpacked.

Own Id: OTP-14189 Aux Id: ERL-348, ERL-349

Improve the Erlang shell's tab-completion of long names.

Own Id: OTP-14200 Aux Id: ERL-352

The reference manual for sys had some faulty information about the 'get_modules' message used by processes where modules change dynamically during runtime. The documentation is now corrected.

Own Id: OTP-14248 Aux Id: ERL-367

Improvements and New Features

Bug fixes, new features and improvements to gen_statem:

A new type init_result/1 has replaced the old init_result/0, so if you used that old type (that was never documented) you have to change your code, which may be regarded as a potential incompatibility.

Changing callback modes after code change did not work since the new callback mode was not recorded. This bug has been fixed.

The event types state_timeout and {call,From} could not be generated with a {next_event,EventType,EventContent} action since they did not pass the runtime type check. This bug has now been corrected.

State entry calls can now be repeated using (new) state callback returns {repeatstate,...}, {repeat_state_and_data,} and repeat_state_and_data.

There have been lots of code cleanup in particular regarding timer handling. For example is async cancel_timer now used. Error handling has also been cleaned up.

To align with probable future changes to the rest of gen_*, terminate/3 has now got a fallback and code_change/4 is not mandatory.

Own Id: OTP-14114

filename:safe_relative_path/1 to sanitize a relative path has been added.

Own Id: OTP-14215

STDLIB 3.2

Fixed Bugs and Malfunctions

When a simple_one_for_one supervisor is shutting down, and a child exits with an exit reason of the form {shutdown, Term}, an error report was earlier printed. This is now corrected.

Own Id: OTP-13907 Aux Id: PR-1158, ERL-163

Allow empty list as parameter of the fun used with dbg:fun2ms/1.

Own Id: OTP-13974

Improvements and New Features

The new behaviour gen_statem has been improved with 3 new features: the possibility to use old style non-proxy timeouts for gen_statem:call/2,3, state entry code, and state timeouts. These are backwards compatible. Minor code and documentation improvements has been performed including a borderline semantics correction of timeout zero handling.

Own Id: OTP-13929 Aux Id: PR-1170, ERL-284

STDLIB 3.1

Fixed Bugs and Malfunctions

The zip:unzip/1,2 and zip:extract/1,2 functions have been updated to handle directory traversal exploits. Any element in the zip file that contains a path that points to a directory above the top level working directory, cwd, will instead be extracted in cwd. An error message is printed for any such element in the zip file during the unzip operation. The keep_old_files option determines if a file will overwrite a previous file with the same name within the zip file.

Own Id: OTP-13633

Correct the contracts for ets:match_object/1,3.

Own Id: OTP-13721 Aux Id: PR-1113

Errors in type specification and Emacs template generation for gen_statem:code_change/4 has been fixed from bugs.erlang.org's Jira cases ERL-172 and ERL-187.

Own Id: OTP-13746 Aux Id: ERL-172, ERL-187

Improvements and New Features

gen_statem has been changed to set the callback mode for a server to what Module:callback_mode/0 returns. This facilitates e.g code downgrade since the callback mode now becomes a property of the currently active code, not of the server process.

Exception handling from Module:init/1 has also been improved.

* POTENTIAL INCOMPATIBILITY *

Own Id: OTP-13752

STDLIB 3.0.1

Fixed Bugs and Malfunctions

Correct a bug regarding typed records in the Erlang shell. The bug was introduced in OTP-19.0.

Own Id: OTP-13719 Aux Id: ERL-182

STDLIB 3.0

Fixed Bugs and Malfunctions

Fix a race bug affecting dets:open_file/2.

Own Id: OTP-13260 Aux Id: seq13002

Don't search for non-existing Map keys twice

For maps:get/2,3 and maps:find/2, searching for an immediate key, e.g. an atom, in a small map, the search was performed twice if the key did not exist.

Own Id: OTP-13459

Avoid stray corner-case math errors on Solaris, e.g. an error is thrown on underflows in exp() and pow() when it shouldn't be.

Own Id: OTP-13531

Fix linting of map key variables

Map keys cannot be unbound and then used in parallel matching.

Example: #{ K := V } = #{ k := K } = M. This is illegal if 'K' is not bound.

Own Id: OTP-13534 Aux Id: ERL-135

Fixed a bug in re on openbsd where sometimes re:run would return an incorrect result.

Own Id: OTP-13602

To avoid potential timer bottleneck on supervisor restart, timer server is no longer used when the supervisor is unable to restart a child.

Own Id: OTP-13618 Aux Id: PR-1001

The Erlang code preprocessor (epp) can handle file names spanning over many tokens. Example: -include("a" "file" "name")..

Own Id: OTP-13662 Aux Id: seq13136

Improvements and New Features

The types of The Abstract Format in the erl_parse module have been refined.

Own Id: OTP-10292

Undocumented syntax for function specifications, -spec F/A :: Domain -> Range, has been removed (without deprecation).

Using the is_subtype(V, T) syntax for constraints (in function specifications) is no longer documented, and the newer syntax V :: T should be used instead. The Erlang Parser still recognizes the is_subtype syntax, and will continue to do so for some time.

* POTENTIAL INCOMPATIBILITY *

Own Id: OTP-11879

The 'random' module has been deprecated. Use the 'rand' module instead.

Own Id: OTP-12502 Aux Id: OTP-12501

Background: In record fields with a type declaration but without an initializer, the Erlang parser inserted automatically the singleton type 'undefined' to the list of declared types, if that value was not present there. That is, the record declaration:

-record(rec, {f1 :: float(), f2 = 42 :: integer(), f3 :: some_mod:some_typ()}).

was translated by the parser to:

-record(rec, {f1 :: float() | 'undefined', f2 = 42 :: integer(), f3 ::
some_mod:some_typ() | 'undefined'}).

The rationale for this was that creation of a "dummy" #rec{} record should not result in a warning from dialyzer that, for example, the implicit initialization of the #rec.f1 field violates its type declaration.

Problems: This seemingly innocent action has some unforeseen consequences.

For starters, there is no way for programmers to declare that e.g. only floats make sense for the f1 field of #rec{} records when there is no "obvious" default initializer for this field. (This also affects tools like PropEr that use these declarations produced by the Erlang parser to generate random instances of records for testing purposes.)

It also means that dialyzer does not warn if e.g. an is_atom/1 test or something more exotic like an atom_to_list/1 call is performed on the value of the f1 field.

Similarly, there is no way to extend dialyzer to warn if it finds record constructions where f1 is not initialized to some float.

Last but not least, it is semantically problematic when the type of the field is an opaque type: creating a union of an opaque and a structured type is very problematic for analysis because it fundamentally breaks the opacity of the term at that point.

Change: To solve these problems the parser will not automatically insert the 'undefined' value anymore; instead the user has the option to choose the places where this value makes sense (for the field) and where it does not and insert the | 'undefined' there manually.

Consequences of this change: This change means that dialyzer will issue a warning for all places where records with uninitialized fields are created and those fields have a declared type that is incompatible with 'undefined' (e.g. float/0). This warning can be suppressed easily by adding | 'undefined' to the type of this field. This also adds documentation that the user really intends to create records where this field is uninitialized.

* POTENTIAL INCOMPATIBILITY *

Own Id: OTP-12719

Remove deprecated functions in the modules erl_scan and erl_parse.

Own Id: OTP-12861

The pre-processor can now expand the ?FUNCTION_NAME and ?FUNCTION_ARITY macros.

Own Id: OTP-13059

A new behaviour gen_statem has been implemented. It has been thoroughly reviewed, is stable enough to be used by at least two heavy OTP applications, and is here to stay. But depending on user feedback, we do not expect but might find it necessary to make minor not backwards compatible changes into OTP-20.0, so its state can be designated as "not quite experimental"...

The gen_statem behaviour is intended to replace gen_fsm for new code. It has the same features and add some really useful:

State code is gathered

The state can be any term

Events can be postponed

Events can be self generated

A reply can be sent from a later state

There can be multiple sys traceable replies

The callback model(s) for gen_statem differs from the one for gen_fsm, but it is still fairly easy to rewrite from gen_fsm to gen_statem.

Own Id: OTP-13065 Aux Id: PR-960

Optimize binary:split/2 and binary:split/3 with native BIF implementation.

Own Id: OTP-13082

Background: The types of record fields have since R12B been put in a separate form by epp:parse_file(), leaving the record declaration form untyped. The separate form, however, does not follow the syntax of type declarations, and parse transforms inspecting -type() attributes need to know about the special syntax. Since the compiler stores the return value of epp:parse_file() as debug information in the abstract code chunk ("Abst" or abstract_code), tools too need to know about the special syntax, if they inspect -type() attributes in abstract code.

Change: No separate type form is created by epp:parse_file(), but the type information is kept in the record fields. This means that all parse transforms and all tools inspecting -record() declarations need to recognize {typed_record_field, Field, Type}.

* POTENTIAL INCOMPATIBILITY *

Own Id: OTP-13148

Unsized fields of the type bytes in binary generators are now forbidden. (The other ways of writing unsized fields, such as binary, are already forbidden.)

Own Id: OTP-13152

The type map/0 is built-in, and cannot be redefined.

Own Id: OTP-13153

Let dets:open_file() exit with a badarg message if given a raw file name (a binary).

Own Id: OTP-13229 Aux Id: ERL-55

Add filename:basedir/2,3

basedir returns suitable path(s) for 'user_cache', 'user_config', 'user_data', 'user_log', 'site_config' and 'site_data'. On linux and linux like systems the paths will respect the XDG environment variables.

Own Id: OTP-13392

There are new preprocessor directives -error(Term) and -warning(Term) to cause a compilation error or a compilation warning, respectively.

Own Id: OTP-13476

Optimize '++' operator and lists:append/2 by using a single pass to build a new list while checking for properness.

Own Id: OTP-13487

Add maps:update_with/3,4 and maps:take/2

Own Id: OTP-13522 Aux Id: PR-1025

lists:join/2 has been added. Similar to string:join/2 but works with arbitrary lists.

Own Id: OTP-13523

Obfuscate asserts to make Dialyzer shut up.

Own Id: OTP-13524 Aux Id: PR-1002

Supervisors now explicitly add their callback module in the return from sys:get_status/1,2. This is to simplify custom supervisor implementations. The Misc part of the return value from sys:get_status/1,2 for a supervisor is now:

[{data, [{"State", State}]},{supervisor,[{"Callback",Module}]}]

* POTENTIAL INCOMPATIBILITY *

Own Id: OTP-13619 Aux Id: PR-1000

Relax translation of initial calls in proc_lib, i.e. remove the restriction to only do the translation for gen_server and gen_fsm. This enables user defined gen based generic callback modules to be displayed nicely in c:i() and observer.

Own Id: OTP-13623

The function queue:lait/1 (misspelling of liat/1) is now deprecated.

Own Id: OTP-13658

STDLIB 2.8.0.1

Improvements and New Features

List subtraction (The -- operator) will now yield properly on large inputs.

Own Id: OTP-15371

STDLIB 2.8

Fixed Bugs and Malfunctions

Fix evaluation in matching of bound map key variables in the interpreter.

Prior to this patch, the following code would not evaluate: X = key,(fun(#{X := value}) -> true end)(#{X => value})

Own Id: OTP-13218

Fix erl_eval not using non-local function handler.

Own Id: OTP-13228 Aux Id: ERL-32

The Erlang Code Linter no longer crashes if there is a -deprecated() attribute but no -module() declaration.

Own Id: OTP-13230 Aux Id: ERL-62

The timestamp in the result returned by dets:info(Tab, safe_fixed) was unintentionally broken as a result of the time API rewrites in OTP 18.0. This has now been fixed.

Own Id: OTP-13239 Aux Id: OTP-11997

A rare race condition in beam_lib when using encrypted abstract format has been eliminated.

Own Id: OTP-13278

Improved maps:with/2 and maps:without/2 algorithms

The new implementation speeds up the execution significantly for all sizes of input.

Own Id: OTP-13376

Improvements and New Features

Time warp safety improvements.

Introduced the options monotonic_timestamp, and strict_monotonic_timestamp to the trace, sequential trace, and system profile functionality. This since the already existing timestamp option is not time warp safe.

Introduced the option safe_fixed_monotonic_time to ets:info/2 and dets:info/2. This since the already existing safe_fixed option is not time warp safe.

Own Id: OTP-13222 Aux Id: OTP-11997

In the shell Ctrl+W (delete word) will no longer consider "." as being part of a word.

Own Id: OTP-13281

STDLIB 2.7

Fixed Bugs and Malfunctions

The Erlang Pretty Printer uses :: for function type constraints.

A bug concerning pretty printing of annotated type union elements in map pair types has been fixed.

Some minor issues regarding the documentation of types and specs have been corrected.

Own Id: OTP-13084

The shell command rp prints strings as lists of integers if pretty printing of lists is set to false.

Own Id: OTP-13145

The shell would crash if a bit syntax expression with conflicting types were given (e.g. if a field type was given as 'integer-binary'). (Thanks to Aleksei Magusev for reporting this bug.)

Own Id: OTP-13157

The rand:export_seed/0 would never return 'undefined' even if no seed has previously been created. Fixed to return 'undefined' if there is no seed in the process dictionary.

Own Id: OTP-13162

Improvements and New Features

Add support for the Delete, Home and End keys in the Erlang shell.

Own Id: OTP-13032

beam_lib:all_chunks/1 and beam_lib:build_module/1 have been documented.

Own Id: OTP-13063

STDLIB 2.6

Fixed Bugs and Malfunctions

In OTP 18.0, qlc does not handle syntax errors well. This bug has been fixed.

Own Id: OTP-12946

Optimize zip:unzip/2 when uncompressing to memory.

Own Id: OTP-12950

The STDLIB reference manual is updated to show correct information about the return value of gen_fsm:reply/2.

Own Id: OTP-12973

re:split2,3 and re:replace/3,4 now correctly handles pre-compiled patterns that have been compiled using the 'unicode' option.

Own Id: OTP-12977

Export shell:catch_exception/1 as documented.

Own Id: OTP-12990

Improvements and New Features

A mechanism for limiting the amount of text that the built-in error logger events will produce has been introduced. It is useful for limiting both the size of log files and the CPU time used to produce them.

This mechanism is experimental in the sense that it may be changed if it turns out that it does not solve the problem it is supposed to solve. In that case, there may be backward incompatible improvements to this mechanism.

See the documentation for the config parameter error_logger_format_depth in the Kernel application for information about how to turn on this feature.

Own Id: OTP-12864

STDLIB 2.5

Fixed Bugs and Malfunctions

Fix handling of single dot in filename:join/2

The reference manual says that filename:join(A,B) is equivalent to filename:join([A,B]). In some rare cases this turns out not to be true. For example:

filename:join("/a/.","b") -> "/a/./b" vs filename:join(["/a/.","b"]) -> "/a/b".

This has been corrected. A single dot is now only kept if it occurs at the very beginning or the very end of the resulting path.

* POTENTIAL INCOMPATIBILITY *

Own Id: OTP-12158

The undocumented option generic_debug for gen_server has been removed.

Own Id: OTP-12183

erl_lint:icrt_export/4 has been rewritten to make the code really follow the scoping rules of Erlang, and not just in most situations by accident.

Own Id: OTP-12186

Add 'trim_all' option to binary:split/3

This option can be set to remove _ALL_ empty parts of the result of a call to binary:split/3.

Own Id: OTP-12301

Correct orddict(3) regarding evaluation order of fold() and map/0.

Own Id: OTP-12651 Aux Id: seq12832

Correct maps module error exceptions

Bad input to maps module function will now yield the following exceptions:

{badmap, NotMap}, or

badarg.

Own Id: OTP-12657

It is now possible to paste text in JCL mode (using Ctrl-Y) that has been copied in the previous shell session. Also a bug that caused the JCL mode to crash when pasting text has been fixed.

Own Id: OTP-12673

Add uptime() shell command.

Own Id: OTP-12752

Cache nowarn_bif_clash functions in erl_lint.

This patch stores nowarn_bif_clash in the lint record. By using erlc +'{eprof,lint_module}' when compiling the erlang parser, we noticed the time spent on nowarn_function/2 reduced from 30% to 0.01%.

Own Id: OTP-12754

Optimize the Erlang Code Linter by using the cached filename information.

Own Id: OTP-12772

If a child of a simple_one_for_one returns ignore from its start function no longer store the child for any restart type. It is not possible to restart or delete the child because the supervisor is a simple_one_for_one.

Own Id: OTP-12793

Make ets:file2tab preserve enabled read_concurrency and write_concurrency options for tables.

Own Id: OTP-12814

There are many cases where user code needs to be able to distinguish between a socket that was closed normally and one that was aborted. Setting the option {show_econnreset, true} enables the user to receive ECONNRESET errors on both active and passive sockets.

Own Id: OTP-12841

Improvements and New Features

Allow maps for supervisor flags and child specs

Earlier, supervisor flags and child specs were given as tuples. While this is kept for backwards compatibility, it is now also allowed to give these parameters as maps, see sup_flags and child_spec.

Own Id: OTP-11043

A new system message, terminate, is added. This can be sent with sys:terminate/2,3. If the receiving process handles system messages properly it will terminate shortly after receiving this message.

The new function proc_lib:stop/1,3 utilizes this new system message and monitors the receiving process in order to facilitate a synchronous stop mechanism for 'special processes'.

proc_lib:stop/1,3 is used by the following functions:

gen_server:stop/1,3 (new)

gen_fsm:stop/1,3 (new)

gen_event:stop/1,3 (modified to be synchronous)

wx_object:stop/1,3 (new)

Own Id: OTP-11173 Aux Id: seq12353

Remove the pg module, which has been deprecated through OTP-17, is now removed from the STDLIB application. This module has been marked experimental for more than 15 years, and has largely been superseded by the pg2 module from the Kernel application.

Own Id: OTP-11907

New BIF: erlang:get_keys/0, lists all keys associated with the process dictionary. Note: erlang:get_keys/0 is auto-imported.

* POTENTIAL INCOMPATIBILITY *

Own Id: OTP-12151 Aux Id: seq12521

Add three new functions to io_lib-- scan_format/2, unscan_format/1, and build_text/1-- which expose the parsed form of the format control sequences to make it possible to easily modify or filter the input to io_lib:format/2. This can e.g. be used in order to replace unbounded-size control sequences like ~w or ~p with corresponding depth-limited ~W and ~P before doing the actual formatting.

Own Id: OTP-12167

Introduce the erl_anno module, an abstraction of the second element of tokens and tuples in the abstract format.

Own Id: OTP-12195

Support variables as Map keys in expressions and patterns

Erlang will accept any expression as keys in Map expressions and it will accept literals or bound variables as keys in Map patterns.

Own Id: OTP-12218

The last traces of Mnemosyne Rules have been removed.

Own Id: OTP-12257

Properly support maps in match_specs

Own Id: OTP-12270

New function ets:take/2. Works the same as ets:delete/2 but also returns the deleted object(s).

Own Id: OTP-12309

string:tokens/2 is somewhat faster, especially if the list of separators only contains one separator character.

Own Id: OTP-12422 Aux Id: seq12774

The documentation of the Abstract Format (in the ERTS User's Guide) has been updated with types and specification. (Thanks to Anthony Ramine.)

The explicit representation of parentheses used in types of the abstract format has been removed. Instead the new functions erl_parse:type_inop_prec() and erl_parse:type_preop_prec() can be used for inserting parentheses where needed.

Own Id: OTP-12492

Prevent zip:zip_open/[12] from leaking file descriptors if parent process dies.

Own Id: OTP-12566

Add a new random number generator, see rand module. It have better characteristics and an improved interface.

Own Id: OTP-12586 Aux Id: OTP-12501, OTP-12502

filename:split/1 when given an empty binary will now return an empty list, to make it consistent with return value when given an empty list.

Own Id: OTP-12716

Add sync option to ets:tab2file/3.

Own Id: OTP-12737 Aux Id: seq12805

Add functions gb_sets:iterator_from() and gb_trees:iterator_from(). (Thanks to Kirill Kinduk.)

Own Id: OTP-12742

Add maps:filter/2 to maps module.

Own Id: OTP-12745

Change some internal data structures to Maps in order to speed up compilation time. Measured speed up is around 10%-15%.

Own Id: OTP-12774

Update orddict to use parameterized types and specs. (Thanks to UENISHI Kota.)

Own Id: OTP-12785

The assert macros in eunit has been moved out to stdlib/include/assert.hrl. This files get included by eunit.hrl. Thus, nothing changes for eunit users, but the asserts can now also be included separately.

Own Id: OTP-12808

STDLIB 2.4

Fixed Bugs and Malfunctions

Behaviour of character types \d, \w and \s has always been to not match characters with value above 255, not 128, i.e. they are limited to ISO-Latin-1 and not ASCII

Own Id: OTP-12521

Improvements and New Features

c:m/1 now displays the module's MD5 sum.

Own Id: OTP-12500

Make ets:i/1 handle binary input from IO server.

Own Id: OTP-12550

STDLIB 2.3

Fixed Bugs and Malfunctions

The documentation of string:tokens/2 now explicitly specifies that adjacent separator characters do not give any empty strings in the resulting list of tokens.

Own Id: OTP-12036

Fix broken deprecation warnings in ssh application

Own Id: OTP-12187

Maps: Properly align union typed assoc values in documentation

Own Id: OTP-12190

Fix filelib:wildcard/2 when 'Cwd' ends with a dot

Own Id: OTP-12212

Allow Name/Arity syntax in maps values inside attributes.

Own Id: OTP-12213

Fix edlin to correctly save text killed with ctrl-u. Prior to this fix, entering text into the Erlang shell and then killing it with ctrl-u followed by yanking it back with ctrl-y would result in the yanked text being the reverse of the original killed text.

Own Id: OTP-12224

If a callback function was terminated with exit/1, there would be no stack trace in the ERROR REPORT produced by gen_server. This has been corrected.

To keep the backwards compatibility, the actual exit reason for the process is not changed.

Own Id: OTP-12263 Aux Id: seq12733

Warnings produced by ms_transform could point out the wrong line number.

Own Id: OTP-12264

Improvements and New Features

Supports tar file creation on other media than file systems mounted on the local machine.

The erl_tar api is extended with erl_tar:init/3 that enables usage of user provided media storage routines. A ssh-specific set of such routines is hidden in the new function ssh_sftp:open_tar/3 to simplify creating a tar archive on a remote ssh server.

A chunked file reading option is added to erl_tar:add/3,4 to save memory on e.g small embedded systems. The size of the slices read from a file in that case can be specified.

Own Id: OTP-12180 Aux Id: seq12715

I/O requests are optimized for long message queues in the calling process.

Own Id: OTP-12315

STDLIB 2.2

Fixed Bugs and Malfunctions

The type spec of the FormFunc argument to sys:handle_debug/4 was erroneously pointing to dbg_fun(). This is now corrected and the new type is format_fun().

Own Id: OTP-11800

Behaviors such as gen_fsm and gen_server should always invoke format_status/2 before printing the state to the logs.

Own Id: OTP-11967

The documentation of dets:insert_new/2 has been corrected. (Thanks to Alexei Sholik for reporting the bug.)

Own Id: OTP-12024

Printing a term with io_lib:format and control sequence w, precision P and field width F, where F< P would fail in one of the two following ways:

If P < printed length of the term, an infinite loop would be entered, consuming all available memory.

If P >= printed length of the term, an exception would be raised.

These two problems are now corrected.

Own Id: OTP-12041

The documentation of maps:values/1 has been corrected.

Own Id: OTP-12055

Expand shell functions in map expressions.

Own Id: OTP-12063

Improvements and New Features

Add maps:with/2

Own Id: OTP-12137

STDLIB 2.1.1

Fixed Bugs and Malfunctions

OTP-11850 fixed filelib:wildcard/1 to work with broken symlinks. This correction, however, introduced problems since symlinks were no longer followed for functions like filelib:ensure_dir/1, filelib:is_dir/1, filelib:file_size/1, etc. This is now corrected.

Own Id: OTP-12054 Aux Id: seq12660

STDLIB 2.1

Fixed Bugs and Malfunctions

filelib:wildcard("broken_symlink") would return an empty list if "broken_symlink" was a symlink that did not point to an existing file.

Own Id: OTP-11850 Aux Id: seq12571

erl_tar can now handle files names that contain Unicode characters. See "UNICODE SUPPORT" in the documentation for erl_tar.

When creating a tar file, erl_tar would sometime write a too short end of tape marker. GNU tar would correctly extract files from such tar file, but would complain about "A lone zero block at...".

Own Id: OTP-11854

When redefining and exporting the type map/0 the Erlang Code Linter (erl_lint) erroneously emitted an error. This bug has been fixed.

Own Id: OTP-11872

Fix evaluation of map updates in the debugger and erl_eval

Reported-by: José Valim

Own Id: OTP-11922

Improvements and New Features

The following native functions now bump an appropriate amount of reductions and yield when out of reductions:

erlang:binary_to_list/1

erlang:binary_to_list/3

erlang:bitstring_to_list/1

erlang:list_to_binary/1

erlang:iolist_to_binary/1

erlang:list_to_bitstring/1

binary:list_to_bin/1

Characteristics impact:

Performance - The functions converting from lists got a performance loss for very small lists, and a performance gain for very large lists.

Priority - Previously a process executing one of these functions effectively got an unfair priority boost. This priority boost depended on the input size. The larger the input was, the larger the priority boost got. This unfair priority boost is now lost.

Own Id: OTP-11888

Add maps:get/3 to maps module. The function will return the supplied default value if the key does not exist in the map.

Own Id: OTP-11951

STDLIB 2.0

Fixed Bugs and Malfunctions

The option dupnames did not work as intended in re. When looking for names with {capture, [Name, ...]}, re:run returned a random instance of the match for that name, instead of the leftmost matching instance, which was what the documentation stated. This is now corrected to adhere to the documentation. The option {capture,all_names} along with a re:inspect/2 function is also added to further help in using named subpatterns.

Own Id: OTP-11205

If option 'binary' was set for standard_input, then c:i() would hang if the output was more than one page long - i.e. then input after "(c)ontinue (q)uit -->" could not be read. This has been corrected. (Thanks to José Valim)

Own Id: OTP-11589

stdlib/lists: Add function droplast/1 This functions drops the last element of a non-empty list. lists:last/1 and lists:droplast/1 are the dual of hd/1 and tl/1 but for the end of a list. (Thanks to Hans Svensson)

Own Id: OTP-11677

Allow all auto imports to be suppressed at once. Introducing the no_auto_import attribute: -compile(no_auto_import). Useful for code generation tools that always use the qualified function names and want to avoid the auto imported functions clashing with local ones. (Thanks to José Valim.)

Own Id: OTP-11682

supervisor_bridge does no longer report normal termination of children. The reason is that in some cases, for instance when the restart strategy is simple_one_for_one, the log could be completely overloaded with reports about normally terminating processes. (Thanks to Artem Ocheredko)

Own Id: OTP-11685

The type annotations for alternative registries using the {via, Module, Name} syntax for sup_name() and sup_ref() in the supervisor module are now consistent with the documentation. Dialyzer should no longer complain about valid supervisor:start_link() and supervisor:start_child() calls. (Thanks to Caleb Helbling.)

Own Id: OTP-11707

Two Dets bugs have been fixed. When trying to open a short file that is not a Dets file, the file was deleted even with just read access. Calling dets:is_dets_file/1 with a file that is not a Dets file, a file descriptor was left open. (Thanks to Håkan Mattsson for reporting the bugs.)

Own Id: OTP-11709

Fix race bug in ets:all. Concurrent creation of tables could cause other tables to not be included in the result. (Thanks to Florian Schintke for bug report)

Own Id: OTP-11726

erl_eval now properly evaluates '=='/2 when it is used in guards. (Thanks to José Valim)

Own Id: OTP-11747

Calls to proplists:get_value/3 are replaced by the faster lists:keyfind/3 in io_lib_pretty. Elements in the list are always 2-tuples. (Thanks to Andrew Thompson)

Own Id: OTP-11752

A qlc bug where filters were erroneously optimized away has been fixed. Thanks to Sam Bobroff for reporting the bug.

Own Id: OTP-11758

A number of compiler errors where unusual or nonsensical code would crash the compiler have been reported by Ulf Norell and corrected by Anthony Ramine.

Own Id: OTP-11770

Since Erlang/OTP R16B the Erlang Core Linter (erl_lint) has not emitted errors when built-in types were re-defined. This bug has been fixed. (Thanks to Roberto Aloi.)

Own Id: OTP-11772

The functions sys:get_state/1,2 and sys:replace_state/2,3 are fixed so they can now be run while the process is sys suspended. To accomplish this, the new callbacks Mod:system_get_state/1 and Mod:system_replace_state/2 are added, which are also implemented by the generic behaviours gen_server, gen_event and gen_fsm.

The potential incompatibility refers to:

The previous behaviour of intercepting the system message and passing a tuple of size 2 as the last argument to sys:handle_system_msg/6 is no longer supported.

The error handling when StateFun in sys:replace_state/2,3 fails is changed from being totally silent to possibly (if the callback module does not catch) throw an exception in the client process.

(Thanks to James Fish and Steve Vinoski)

* POTENTIAL INCOMPATIBILITY *

Own Id: OTP-11817

Improvements and New Features

Options to set match_limit and match_limit_recursion are added to re:run. The option report_errors is also added to get more information when re:run fails due to limits or compilation errors.

Own Id: OTP-10285

The pre-defined types array/0, dict/0, digraph/0, gb_set/0, gb_tree/0, queue/0, set/0, and tid/0 have been deprecated. They will be removed in Erlang/OTP 18.0.

Instead the types array:array/0, dict:dict/0, digraph:graph/0, gb_set:set/0, gb_tree:tree/0, queue:queue/0, sets:set/0, and ets:tid/0 can be used. (Note: it has always been necessary to use ets:tid/0.)

It is allowed in Erlang/OTP 17.0 to locally re-define the types array/0, dict/0, and so on.

New types array:array/1, dict:dict/2, gb_sets:set/1, gb_trees:tree/2, queue:queue/1, and sets:set/1 have been added.

A compiler option, nowarn_deprecated_type, has been introduced. By including the attribute

-compile(nowarn_deprecated_type).

in an Erlang source file, warnings about deprecated types can be avoided in Erlang/OTP 17.0.

The option can also be given as a compiler flag:

erlc +nowarn_deprecated_type file.erl

Own Id: OTP-10342

Calls to erlang:open_port/2 with 'spawn' are updated to handle space in the command path.

Own Id: OTP-10842

Dialyzer's unmatched_return warnings have been corrected.

Own Id: OTP-10908

Forbid unsized fields in patterns of binary generators and simplified v3_core's translation of bit string generators. (Thanks to Anthony Ramine.)

Own Id: OTP-11186

The version of the PCRE library Used by Erlang's re module is raised to 8.33 from 7.6. This means, among other things, better Unicode and Unicode Character Properties support. New options connected to PCRE 8.33 are also added to the re module (ucd, notempty_atstart, no_start_optimize). PCRE has extended the regular expression syntax between 7.6 and 8.33, why this imposes a potential incompatibility. Only very complicated regular expressions may be affected, but if you know you are using obscure features, please test run your regular expressions and verify that their behavior has not changed.

* POTENTIAL INCOMPATIBILITY *

Own Id: OTP-11204

Added dict:is_empty/1 and orddict:is_empty/1. (Thanks to Magnus Henoch.)

Own Id: OTP-11353

A call to either the garbage_collect/1 BIF or the check_process_code/2 BIF may trigger garbage collection of another processes than the process calling the BIF. The previous implementations performed these kinds of garbage collections without considering the internal state of the process being garbage collected. In order to be able to more easily and more efficiently implement yielding native code, these types of garbage collections have been rewritten. A garbage collection like this is now triggered by an asynchronous request signal, the actual garbage collection is performed by the process being garbage collected itself, and finalized by a reply signal to the process issuing the request. Using this approach processes can disable garbage collection and yield without having to set up the heap in a state that can be garbage collected.

The garbage_collect/2, and check_process_code/3 BIFs have been introduced. Both taking an option list as last argument. Using these, one can issue asynchronous requests.

code:purge/1 and code:soft_purge/1 have been rewritten to utilize asynchronous check_process_code requests in order to parallelize work.

Characteristics impact: A call to the garbage_collect/1 BIF or the check_process_code/2 BIF will normally take longer time to complete while the system as a whole wont be as much negatively effected by the operation as before. A call to code:purge/1 and code:soft_purge/1 may complete faster or slower depending on the state of the system while the system as a whole wont be as much negatively effected by the operation as before.

Own Id: OTP-11388 Aux Id: OTP-11535, OTP-11648

Improve the documentation of the supervisor's via reference. (Thanks to MaximMinin.)

Own Id: OTP-11399

orddict:from_list/1 now uses the optimized sort routines in the lists module instead of (essentially) an insertion sort. Depending on the input data, the speed of the new from_list/1 is anything from slightly faster up to several orders of magnitude faster than the old from_list/1.

(Thanks to Steve Vinoski.)

Own Id: OTP-11552

EEP43: New data type - Maps

With Maps you may for instance:

____ - M0 = #{ a => 1, b => 2}, % create associations

____ - M1 = M0#{ a := 10 }, % update values

____ - M2 = M1#{ "hi" => "hello"}, % add new associations

____ - #{ "hi" := V1, a := V2, b := V3} = M2. % match keys with values

For information on how to use Maps please see Map Expressions in the Reference Manual.

The current implementation is without the following features:

____ - No variable keys

____ - No single value access

____ - No map comprehensions

Note that Maps is experimental during OTP 17.0.

Own Id: OTP-11616

When tab completing the erlang shell now expands zero-arity functions all the way to closing parenthesis, unless there is another function with the same name and a different arity. (Thanks to Pierre Fenoll.)

Own Id: OTP-11684

The Erlang Code Preprocessor (epp) could loop when encountering a circular macro definition in an included file. This bug has been fixed.

Thanks to Maruthavanan Subbarayan for reporting the bug, and to Richard Carlsson for providing a bug fix.

Own Id: OTP-11728

The Erlang Code Linter (erl_lint) has since Erlang/OTP R13B emitted warnings whenever any of the types arity/0, bitstring/0, iodata/0, or boolean/0 were re-defined. Now errors are emitted instead.

* POTENTIAL INCOMPATIBILITY *

Own Id: OTP-11771

The encoding option of erl_parse:abstract/2 has been extended to include none and a callback function (a predicate).

Own Id: OTP-11807

Export zip option types to allow referal from other modules.

Thanks to Pierre Fenoll and Håkan Mattson

Own Id: OTP-11828

The module pg has been deprecated and will be removed in Erlang/OTP 18.

Own Id: OTP-11840

STDLIB 1.19.4

Fixed Bugs and Malfunctions

Fix typo in gen_server.erl. Thanks to Brian L. Troutwine.

Own Id: OTP-11398

Spec for atan2 should be atan2(Y, X), not atan2(X, Y). Thanks to Ary Borenszweig.

Own Id: OTP-11465

Improvements and New Features

Add XML marker for regexp syntax. Thanks to Håkan Mattson.

Own Id: OTP-11442

STDLIB 1.19.3

Fixed Bugs and Malfunctions

The functions dets:foldl/3, dets:foldr/3, and dets:traverse/2 did not release the table after having traversed the table to the end. The bug was introduced in R16B. (Thanks to Manuel Duran Aguete.)

Own Id: OTP-11245

If the fun M:F/A construct was used erroneously the linter could crash. (Thanks to Mikhail Sobolev.)

Own Id: OTP-11254

The specifications of io_lib:fread/2,3 have been corrected. (Thanks to Chris King and Kostis Sagonas for pinpointing the bug.)

Own Id: OTP-11261

Improvements and New Features

Fixed type typo in gen_server.

Own Id: OTP-11200

Update type specs in filelib and io_prompt. Thanks to Jose Valim.

Own Id: OTP-11208

Fix typo in abcast() function comment. Thanks to Johannes Weissl.

Own Id: OTP-11219

Make edlin understand a few important control keys. Thanks to Stefan Zegenhagen.

Own Id: OTP-11251

Export the edge/0 type from the digraph module. Thanks to Alex Ronne Petersen.

Own Id: OTP-11266

Fix variable usage tracking in erl_lint and fixed unsafe variable tracking in try expressions. Thanks to Anthony Ramine.

Own Id: OTP-11268

STDLIB 1.19.2

Fixed Bugs and Malfunctions

The Erlang scanner no longer accepts floating point numbers in the input string.

Own Id: OTP-10990

When converting a faulty binary to a list with unicode:characters_to_list, the error return value could contain a faulty "rest", i.e. the io_list of characters that could not be converted was wrong. This happened only if input was a sub binary and conversion was from utf8. This is now corrected.

Own Id: OTP-11080

The type hook_function() has been corrected in erl_pp, the Erlang Pretty Printer.

The printing of invalid forms, e.g. record field types, has also been fixed. It has been broken since R16B.

(Thanks to Tomáš Janoušek.)

Own Id: OTP-11100

Fix receive support in erl_eval with a BEAM module. Thanks to Anthony Ramine.

Own Id: OTP-11137

Improvements and New Features

Delete obsolete note about simple-one-for-one supervisor. Thanks to Magnus Henoch.

Own Id: OTP-10938

When selecting encoding of a script written in Erlang (escript) the optional directive on the second line is now recognized.

Own Id: OTP-10951

The function erl_parse:abstract/2 has been documented.

Own Id: OTP-10992

Integrate elliptic curve contribution from Andreas Schultz

In order to be able to support elliptic curve cipher suites in SSL/TLS, additions to handle elliptic curve infrastructure has been added to public_key and crypto.

This also has resulted in a rewrite of the crypto API to gain consistency and remove unnecessary overhead. All OTP applications using crypto has been updated to use the new API.

Impact: Elliptic curve cryptography (ECC) offers equivalent security with smaller key sizes than other public key algorithms. Smaller key sizes result in savings for power, memory, bandwidth, and computational cost that make ECC especially attractive for constrained environments.

Own Id: OTP-11009

Added sys:get_state/1,2 and sys:replace_state/2,3. Thanks to Steve Vinoski.

Own Id: OTP-11013

Optimizations to gen mechanism. Thanks to Loïc Hoguin.

Own Id: OTP-11025

Optimizations to gen.erl. Thanks to Loïc Hoguin.

Own Id: OTP-11035

Use erlang:demonitor(Ref, [flush]) where applicable. Thanks to Loïc Hoguin.

Own Id: OTP-11039

Erlang source files with non-ASCII characters are now encoded in UTF-8 (instead of latin1).

Own Id: OTP-11041 Aux Id: OTP-10907

Fix rest_for_one and one_for_all restarting a child not terminated. Thanks to James Fish.

Own Id: OTP-11042

Fix excessive CPU consumption of timer_server. Thanks to Aliaksey Kandratsenka.

Own Id: OTP-11053

Rename and document lists:zf/2 as lists:filtermap/2. Thanks to Anthony Ramine.

Own Id: OTP-11078

Fixed an inconsistent state in epp. Thanks to Anthony Ramine

Own Id: OTP-11079

c:ls(File) will now print File, similar to ls(1) in Unix. The error messages have also been improved. (Thanks to Bengt Kleberg.)

Own Id: OTP-11108

Support callback attributes in erl_pp. Thanks to Anthony Ramine.

Own Id: OTP-11140

Improve erl_lint performance. Thanks to José Valim.

Own Id: OTP-11143

STDLIB 1.19.1

Fixed Bugs and Malfunctions

Bugs related to Unicode have been fixed in the erl_eval module.

Own Id: OTP-10622 Aux Id: kunagi-351 [262]

filelib:wildcard("some/relative/path/*.beam", Path) would fail to match any file. That is, filelib:wildcard/2 would not work if the first component of the pattern did not contain any wildcard characters. (A previous attempt to fix the problem in R15B02 seems to have made matters worse.)

(Thanks to Samuel Rivas and Tuncer Ayaz.)

There is also an incompatible change to the Path argument. It is no longer allowed to be a binary.

* POTENTIAL INCOMPATIBILITY *

Own Id: OTP-10812

Improvements and New Features

The new STDLIB application variable shell_strings can be used for determining how the Erlang shell outputs lists of integers. The new function shell:strings/1 toggles the value of the variable.

The control sequence modifier l can be used for turning off the string recognition of ~p and ~P.

Own Id: OTP-10755

Miscellaneous updates due to Unicode support.

Own Id: OTP-10820

Extend ~ts to handle binaries with characters coded in ISO-latin-1

Own Id: OTP-10836

The +pc flag to erl can be used to set the range of characters considered printable. This affects how the shell and io:format("~tp",...) functionality does heuristic string detection. More can be read in STDLIB users guide.

Own Id: OTP-10884

STDLIB 1.19

Fixed Bugs and Malfunctions

Wildcards such as "some/path/*" passed to filelib:wildcard/2 would fail to match any file. (Thanks to Samuel Rivas for reporting this bug.)

Own Id: OTP-6874 Aux Id: kunagi-190 [101]

Fixed error handling in proc_lib:start which could hang if the spawned process died in init.

Own Id: OTP-9803 Aux Id: kunagi-209 [120]

Allow ** in filelib:wildcard

Two adjacent * used as a single pattern will match all files and zero or more directories and subdirectories. (Thanks to José Valim)

Own Id: OTP-10431

Add the \gN and \g{N} syntax for back references in re:replace/3,4 to allow use with numeric replacement strings. (Thanks to Vance Shipley)

Own Id: OTP-10455

Export ets:match_pattern/0 type (Thanks to Joseph Wayne Norton)

Own Id: OTP-10472

Fix printing the empty binary at depth 1 with ~W (Thanks to Andrew Thompson)

Own Id: OTP-10504

The type ascii_string() in the base64 module has been corrected. The type file:file_info() has been cleaned up. The type file:fd() has been made opaque in the documentation.

Own Id: OTP-10624 Aux Id: kunagi-352 [263]

Improvements and New Features

Dets tables are no longer fixed while traversing with a bound key (when only the objects with the right key are matched). This optimization affects the functions match/2, match_object/2, select/2, match_delete/2, and select_delete/2.

Own Id: OTP-10097

Support for Unicode has been implemented.

Own Id: OTP-10302

The linter now warns for opaque types that are not exported, as well as for under-specified opaque types.

Own Id: OTP-10436

The type file:name() has been substituted for the type file:filename() in the following functions in the filename module: absname/2, absname_join/2, join/1,2, and split/1.

Own Id: OTP-10474

If a child process fails in its start function, then the error reason was earlier only reported as an error report from the error_handler, and supervisor:start_link would only return {error,shutdown}. This has been changed so the supervisor will now return {error,{shutdown,Reason}}, where Reason identifies the failing child and its error reason. (Thanks to Tomas Pihl)

* POTENTIAL INCOMPATIBILITY *

Own Id: OTP-10490

Where necessary a comment stating encoding has been added to Erlang files. The comment is meant to be removed in Erlang/OTP R17B when UTF-8 becomes the default encoding.

Own Id: OTP-10630

The contracts and types of the modules erl_scan and sys have been corrected and improved. (Thanks to Kostis Sagonas.)

Own Id: OTP-10658

The Erlang shell now skips the rest of the line when it encounters an Erlang scanner error.

Own Id: OTP-10659

Clean up some specs in the proplists module. (Thanks to Kostis Sagonas.)

Own Id: OTP-10663

Some examples overflowing the width of PDF pages have been corrected.

Own Id: OTP-10665

Enable escript to accept emulator arguments when script file has no shebang. Thanks to Magnus Henoch

Own Id: OTP-10691

Fix bug in queue:out/1, queue:out_r/1 that makes it O(N^2) in worst case. Thanks to Aleksandr Erofeev.

Own Id: OTP-10722

There are new functions in the epp module which read the character encoding from files. See epp for more information.

Own Id: OTP-10742 Aux Id: OTP-10302

The functions in io_lib have been adjusted for Unicode. The existing functions write_string() and so on now take Unicode strings, while the old behavior has been taken over by new functions write_latin1_string() and so on. There are also new functions to write Unicode strings as Latin-1 strings, mainly targetted towards the Erlang pretty printer (erl_pp).

Own Id: OTP-10745 Aux Id: OTP-10302

The new functions proc_lib:format/2 and erl_parse:abstract/2 accept an encoding as second argument.

Own Id: OTP-10749 Aux Id: OTP-10302

Increased potential concurrency in ETS for write_concurrency option. The number of internal table locks has increased from 16 to 64. This makes it four times less likely that two concurrent processes writing to the same table would collide and thereby serialized. The cost is an increased constant memory footprint for tables using write_concurrency. The memory consumption per inserted record is not affected. The increased footprint can be particularly large if write_concurrency is combined with read_concurrency.

Own Id: OTP-10787

STDLIB 1.18.3

Fixed Bugs and Malfunctions

Minor test updates

Own Id: OTP-10591

STDLIB 1.18.2

Fixed Bugs and Malfunctions

Fixed bug where if given an invalid drive letter on windows ensure dir would go into an infinite loop.

Own Id: OTP-10104

Calls to gen_server:enter_loop/4 where ServerName has a global scope and no timeout is given now works correctly.

Thanks to Sam Bobroff for reporting the issue.

Own Id: OTP-10130

fix escript/primary archive reloading

If the mtime of an escript/primary archive file changes after being added to the code path, correctly reload the archive and update the cache. (Thanks to Tuncer Ayaz)

Own Id: OTP-10151

Fix bug that in some cases could cause corrupted binaries in ETS tables with compressed option.

Own Id: OTP-10182

Fix filename:nativename/1 on Win32

Don't choke on paths given as binary argument on Win32. Thanks to Jan Klötzke

Own Id: OTP-10188

Fix bug in ets:test_ms/2 that could cause emulator crash when using '$_' in match spec.

Own Id: OTP-10190

Fix bug where zip archives wrongly have a first disk number set to 1

Own Id: OTP-10223

Improvements and New Features

The message printed by the Erlang shell as an explanation of the badarith error has been corrected. (Thanks to Matthias Lang.)

Own Id: OTP-10054

STDLIB 1.18.1

Fixed Bugs and Malfunctions

References to is_constant/1 (which was removed in the R12 release) has been removed from documentation and code.

Own Id: OTP-6454 Aux Id: seq10407

Leave control back to gen_server during supervisor's restart loop

When an attempt to restart a child failed, supervisor would earlier keep the execution flow and try to restart the child over and over again until it either succeeded or the restart frequency limit was reached. If none of these happened, supervisor would hang forever in this loop.

This commit adds a timer of 0 ms where the control is left back to the gen_server which implements the supervisor. This way any incoming request to the supervisor will be handled - which could help breaking the infinite loop - e.g. shutdown request for the supervisor or for the problematic child.

This introduces some incompatibilities in stdlib due to new return values from supervisor:

restart_child/2 can now return {error,restarting}

delete_child/2 can now return {error,restarting}

which_children/1 returns a list of {Id,Child,Type,Mods}, where Child, in addition to the old pid() or 'undefined', now also can be 'restarting'.

* POTENTIAL INCOMPATIBILITY *

Own Id: OTP-9549

If a temporary child's start function returned 'ignore', then the supervisor would keep it's child specification. This has been corrected. Child specifications for non-existing temporary children shall never be kept.

Own Id: OTP-9782 Aux Id: seq11964

Use universal time as base in error logger

Previous conversion used the deprecated calendar:local_time_to_universal_time/1

Own Id: OTP-9854

Calling a guard test (such as is_list/1) from the top-level in a guard, would cause a compiler crash if there was a local definition with the same name. Corrected to reject the program with an error message.

Own Id: OTP-9866

Fix the type spec from the doc of binary:part/3 (Thanks to Ricardo Catalinas Jiménez)

Own Id: OTP-9920

Correct spelling of registered (Thanks to Richard Carlsson)

Own Id: OTP-9925

Put gb_trees documentation into alphabetical order (Thanks to Aidan Hobson Sayers)

Own Id: OTP-9929

Fix bug in ETS with compressed option and insertion of term containing large integers (>2G) on 64-bit machines. Seen to cause emulator crash. (Thanks to Diego Llarrull for excellent bug report)

Own Id: OTP-9932

Add plugin support for alternative name lookup This patch introduces a new way of locating a behaviour instance: {via, Module, Name}. (Thanks to Ulf Wiger)

Own Id: OTP-9945

The function digraph_utils:condensation/1 used to create a digraph containing loops contradicting the documentation which states that the created digraph is free of cycles. This bug has been fixed. (Thanks to Kostis Sagonas for finding the bug.)

Own Id: OTP-9953

When an escript ends now all printout to standard output and standard error gets out on the terminal. This bug has been corrected by changing the behaviour of erlang:halt/0,1, which should fix the same problem for other escript-like applications, i.e that data stored in the output port driver buffers got lost when printing on a TTY and exiting through erlang:halt/0,1.

The BIF:s erlang:halt/0,1 has gotten improved semantics and there is a new BIF erlang:halt/2 to accomplish something like the old semantics. See the documentation.

Now erlang:halt/0 and erlang:halt/1 with an integer argument will close all ports and allow all pending async threads operations to finish before exiting the emulator. Previously erlang:halt/0 and erlang:halt(0) would just wait for pending async threads operations but not close ports. And erlang:halt/1 with a non-zero integer argument would not even wait for pending async threads operations.

To roughly the old behaviour, to not wait for ports and async threads operations when you exit the emulator, you use erlang:halt/2 with an integer first argument and an option list containing {flush,false} as the second argument. Note that now is flushing not dependant of the exit code, and you cannot only flush async threads operations which we deemed as a strange behaviour anyway.

Also, erlang:halt/1,2 has gotten a new feature: If the first argument is the atom 'abort' the emulator is aborted producing a core dump, if the operating system so allows.

Own Id: OTP-9985

Add escript win32 alternative invocation. escript can now be started as both "escript.exe" and "escript" (Thanks to Pierre Rouleau)

Own Id: OTP-9997

STDLIB 1.18

Fixed Bugs and Malfunctions

Improved algorithm in module random. Avoid seed values that are even divisors of the primes and by that prevent getting sub-seeds that are stuck on zero. Worst case was random:seed(0,0,0) that produced a series of only zeros. This is an incompatible change in the sense that applications that relies on reproducing a specific series for a given seed will fail. The pseudo random output is still deterministic but different compared to earlier versions.

* POTENTIAL INCOMPATIBILITY *

Own Id: OTP-8713

Calls to global:whereis_name/1 have been substituted for calls to global:safe_whereis_name/1 since the latter is not safe at all.

The reason for not doing this earlier is that setting a global lock masked out a bug concerning the restart of supervised children. The bug has now been fixed by a modification of global:whereis_name/1. (Thanks to Ulf Wiger for code contribution.)

A minor race conditions in gen_fsm:start* has been fixed: if one of these functions returned {error, Reason} or ignore, the name could still be registered (either locally or in global. (This is the same modification as was done for gen_server in OTP-7669.)

The undocumented function global:safe_whereis_name/1 has been removed.

Own Id: OTP-9212 Aux Id: seq7117, OTP-4174

If a child of a supervisor terminates with reason {shutdown,Term} it is now handled by the supervisor as if the reason was 'shutdown'.

For children with restart type 'permanent', this implies no change. For children with restart type 'transient', the child will no longer be restarted and no supervisor report will be written. For children with restart type 'temporary', no supervisor report will be written.

* POTENTIAL INCOMPATIBILITY *

Own Id: OTP-9222

Minor improvement of documentation regarding supervisor restart strategy for temporary and transient child processes.

Own Id: OTP-9381

A Dets table with sufficiently large buckets could not always be repaired. This bug has been fixed.

The format of Dets files has been modified. When downgrading tables created with the new system will be repaired. Otherwise the modification should not be noticeable.

Own Id: OTP-9607

A few contracts in the lists module have been corrected.

Own Id: OTP-9616

Add '-callback' attributes in stdlib's behaviours

Replace the behaviour_info(callbacks) export in stdlib's behaviours with -callback' attributes for all the callbacks. Update the documentation with information on the callback attribute Automatically generate 'behaviour_info' function from '-callback' attributes

'behaviour_info(callbacks)' is a special function that is defined in a module which describes a behaviour and returns a list of its callbacks.

This function is now automatically generated using the '-callback' specs. An error is returned by lint if user defines both '-callback' attributes and the behaviour_info/1 function. If no type info is needed for a callback use a generic spec for it. Add '-callback' attribute to language syntax

Behaviours may define specs for their callbacks using the familiar spec syntax, replacing the '-spec' keyword with '-callback'. Simple lint checks are performed to ensure that no callbacks are defined twice and all types referred are declared.

These attributes can be then used by tools to provide documentation to the behaviour or find discrepancies in the callback definitions in the callback module.

Add callback specs into 'application' module in kernel Add callback specs to tftp module following internet documentation Add callback specs to inets_service module following possibly deprecated comments

Own Id: OTP-9621

If a Dets table had been properly closed but the space management data could not been read, it was not possible to repair the file. This bug has been fixed.

Own Id: OTP-9622

The Unicode noncharacter code points 16#FFFE and 16#FFFE were not allowed to be encoded or decoded using the unicode module or bit syntax. That was inconsistent with the other noncharacters 16#FDD0 to 16#FDEF that could be encoded/decoded. To resolve the inconsistency, 16#FFFE and 16#FFFE can now be encoded and decoded. (Thanks to Alisdair Sullivan.)

* POTENTIAL INCOMPATIBILITY *

Own Id: OTP-9624

Make epp search directory of current file first when including another file This completes a partial fix in R11 that only worked for include_lib(). (Thanks to Richard Carlsson)

Own Id: OTP-9645

ms_transform: Fix incorrect `variable shadowed' warnings

This patch removes incorrect passing of variable bindings from one function clause to another. (Thanks to Haitao Li)

Own Id: OTP-9646

Explicitly kill dynamic children in supervisors

According to the supervisor's documentation: "Important note on simple-one-for-one supervisors: The dynamically created child processes of a simple-one-for-one supervisor are not explicitly killed, regardless of shutdown strategy, but are expected to terminate when the supervisor does (that is, when an exit signal from the parent process is received)."

All is fine as long as we stop simple_one_for_one supervisor manually. Dynamic children catch the exit signal from the supervisor and leave. But, if this happens when we stop an application, after the top supervisor has stopped, the application master kills all remaining processes associated to this application. So, dynamic children that trap exit signals can be killed during their cleanup (here we mean inside terminate/2). This is unpredictable and highly time-dependent.

In this commit, supervisor module is patched to explicitly terminate dynamic children accordingly to the shutdown strategy.

NOTE: Order in which dynamic children are stopped is not defined. In fact, this is "almost" done at the same time.

Stack errors when dynamic children are stopped

Because a simple_one_for_one supervisor can have many workers, we stack errors during its shutdown to report only one message for each encountered error type. Instead of reporting the child's pid, we use the number of concerned children. (Thanks to Christopher Faulet)

Own Id: OTP-9647

Allow an infinite timeout to shutdown worker processes

Now, in child specification, the shutdown value can also be set to infinity for worker children. This restriction was removed because this is not always possible to predict the shutdown time for a worker. This is highly application-dependent. Add a warning to docs about workers' shutdown strategy (Thanks to Christopher Faulet)

Own Id: OTP-9648

A badarg would sometimes occur in supervisor when printing error reports and the child pid was undefined. This has been corrected.

Own Id: OTP-9669

Fix re:split spec not to accept option 'global'(Thanks to Shunichi Shinohara)

Own Id: OTP-9691

Improvements and New Features

Fix a few tests that used to fail on the HiPE platform.

Own Id: OTP-9637

Variables are now now allowed in 'fun M:F/A' as suggested by Richard O'Keefe in EEP-23.

The representation of 'fun M:F/A' in the abstract format has been changed in an incompatible way. Tools that directly read or manipulate the abstract format (such as parse transforms) may need to be updated. The compiler can handle both the new and the old format (i.e. extracting the abstract format from a pre-R15 BEAM file and compiling it using compile:forms/1,2 will work). The syntax_tools application can also handle both formats.

* POTENTIAL INCOMPATIBILITY *

Own Id: OTP-9643

Tuple funs (a two-element tuple with a module name and a function) are now officially deprecated and will be removed in R16. Use 'fun M:F/A' instead. To make you aware that your system uses tuple funs, the very first time a tuple fun is applied, a warning will be sent to the error logger.

Own Id: OTP-9649

The deprecated 'regexp' module has been removed. Use the 're' module instead.

* POTENTIAL INCOMPATIBILITY *

Own Id: OTP-9737

filename:find_src/1,2 will now work on stripped BEAM files (reported by Per Hedeland). The HiPE compiler will also work on stripped BEAM files. The BEAM compiler will no longer include compilation options given in the source code itself in M:module_info(compile) (because those options will be applied anyway if the module is re-compiled).

Own Id: OTP-9752

STDLIB 1.17.5

Fixed Bugs and Malfunctions

erl_tar:extract failed when executed inside a directory with some parent directory to which the user has no read access. This has been corrected.

Own Id: OTP-9368

A bug in erl_scan:set_attribute/3 has been fixed.

Own Id: OTP-9412

The contract of io_lib:fread() has been corrected.

Own Id: OTP-9413 Aux Id: seq11873

A crash in iolib:fread/2 when end of input data was encountered while trying to match literal characters, which should return {more,,,} but instead crashed, has been corrected. Reported by Klas Johansson.

A similar peculiarity for io:fread when encountering end of file before any field data has also been corrected.

Own Id: OTP-9439

The contract of timer:now_diff() has been corrected. (Thanks to Alex Morarash).

Own Id: OTP-9450

Fix minor typo in gen_fsm documentation (Thanks to Haitao Li)

Own Id: OTP-9456

The contracts of zip:zip_list_dir/1 and zip:zip_get/2 have been corrected.

Own Id: OTP-9471 Aux Id: seq11887, OTP-9472

A bug in zip:zip_open() has been fixed.

Own Id: OTP-9472 Aux Id: seq11887, OTP-9471

Fix trivial documentation errors(Thanks to Matthias Lang)

Own Id: OTP-9498

Add a proplist() type

Recently I was adding specs to an API and found that there is no canonical proplist() type defined. (Thanks to Ryan Zezeski)

Own Id: OTP-9499

fix supervisors restarting temporary children

In the current implementation of supervisors, temporary children should never be restarted. However, when a temporary child is restarted as part of a one_for_all or rest_for_one strategy where the failing process is not the temporary child, the supervisor still tries to restart it.

Because the supervisor doesn't keep some of the MFA information of temporary children, this causes the supervisor to hit its restart limit and crash.

This patch fixes the behaviour by inserting a clause in terminate_children/2-3 (private function) that will omit temporary children when building a list of killed processes, to avoid having the supervisor trying to restart them again.

Only supervisors in need of restarting children used the list, so the change should be of no impact for the functions that called terminate_children/2-3 only to kill all children.

The documentation has been modified to make this behaviour more explicit. (Thanks to Fred Hebert)

Own Id: OTP-9502

fix broken edoc annotations (Thanks to Richard Carlsson)

Own Id: OTP-9516

XML files have been corrected.

Own Id: OTP-9550 Aux Id: OTP-9541

Handle rare race in the crypto key server functionality

Own Id: OTP-9586

Improvements and New Features

Types and specifications have been added.

Own Id: OTP-9356

The contracts of the queue module have been modified.

Own Id: OTP-9418

Contracts in STDLIB and Kernel have been improved and type errors have been corrected.

Own Id: OTP-9485

Types for several BIFs have been extended/corrected. Also the types for types for lists:keyfind/3, lists:keysearch/3, and lists:keyemember/3 have been corrected. The incorrect/incomplete types could cause false dialyzer warnings.

Own Id: OTP-9496

STDLIB 1.17.4

Fixed Bugs and Malfunctions

The default value undefined was added to records field types in such a way that the result was not always a well-formed type. This bug has been fixed.

Own Id: OTP-9147

Update index file atomically

Since the log_mf_h index file might be read by other processes than the error handler (e.g. by the rb tool), this file should be updated atomically. This will avoid hitting the time gap between opening the file in write mode (and thus emptying the file) and the actual update with the new contents. To do this, a temporary file is written, and the file:rename/1 used to replace the real index file.

Own Id: OTP-9148

Fixed various typos across the documentation (Thanks to Tuncer Ayaz)

Own Id: OTP-9154

Supervisors should not save child-specs for temporary processes when they terminate as they should not be restarted. Saving the temporary child spec will result in that you cannot start a new temporary process with the same child spec as an already terminated temporary process. Since R14B02 you cannot restart a temporary temporary process as arguments are no longer saved, it has however always been semantically incorrect to restart a temporary process. Thanks to Filipe David Manana for reporting this and suggesting a solution.

Own Id: OTP-9167 Aux Id: OTP-9064

Various small documentation fixes (Thanks to Bernard Duggan)

Own Id: OTP-9172

Fix format_status bug for unregistered gen_event processes

Port the gen_fsm code for format_status to gen_event in order to prevent a lists:concat([...,pid()]) crash when calling sys:get_status/1 on an unregistered gen_event process.

Refactor formatstatus header code from gen* behaviours to module gen.

Extend the format_status tests in gen_event_SUITE to cover format_status bugs with anonymous gen_event processes. (Thanks To Geoff Cant)

Own Id: OTP-9218

List of pids changed to 'set' in supervisor for dynamic temporary children. Accessing the list would not scale well when adding/deleting many children. (Thanks to Evgeniy Khramtsov)

Own Id: OTP-9242

Change pool module to attempt to attach to nodes that are already running

The pool module prints out an error message and takes no further action for nodes that are already running. This patch changes that behavior so that if the return from slave:start/3 is {already_running, Node} then an attempt to attach to the node is still made. This makes sense because the node has been specified by the user in the .hosts.erlang file indicating a wish for the node to be part of the pool and a manual attach can be successfully made after the pool is started.(Thanks to Kelly McLaughlin)

Own Id: OTP-9244

unicode: document 16#FFFE and 16#FFFF (non chars)(Thanks to Tuncer Ayaz)

Own Id: OTP-9256

re: remove gratuitous "it " in manpage (Thanks to Tuncer Ayaz)

Own Id: OTP-9307

A bug in erl_eval(3) has been fixed.

Own Id: OTP-9322

Improvements and New Features

Add timer:tc/1 and remove the catch in tc/2 and tc/3. The time measuring functions will thus no longer trap exits, errors or throws caused by the measured function.

* POTENTIAL INCOMPATIBILITY *

Own Id: OTP-9169

Allow supervisor:terminate_child(SupRef,Pid) for simple_one_for_one supervisors

supervisor:terminate_child/2 was earlier not allowed if the supervisor used restart strategy simple_one_for_one. This is now changed so that children of this type of supervisors can be terminated by specifying the child's Pid.

(Thanks to Vance Shipley.)

Own Id: OTP-9201

Types and specifications have been added.

Own Id: OTP-9267

Erlang types and specifications are used for documentation.

Own Id: OTP-9271

Allow Dets tablenames to be arbitrary terms.

Own Id: OTP-9282

A specification that could cause problems for Dialyzer has been fixed. An opaque type in erl_eval has been turned in to a ordinary type. This is a temporary fix.

Own Id: OTP-9333

STDLIB 1.17.3

Fixed Bugs and Malfunctions

Two bugs in io:format for ~F.~Ps has been corrected. When length(S) >= abs(F) > P, the precision P was incorrectly ignored. When F == P > length(S) the result was incorrectly left adjusted. Bug found by Ali Yakout who also provided a fix.

Own Id: OTP-8989 Aux Id: seq11741

Fix exception generation in the io module

Some functions did not generate correct badarg exception on a badarg exception.

Own Id: OTP-9045

Fixes to the dict and orddict module documentation

Fixed grammar and one inconsistency (Key - Value instead of key/value, since everywhere else the former is used). (thanks to Filipe David Manana)

Own Id: OTP-9083

Add ISO week number calculation functions to the calendar module in stdlib

This new feature adds the missing week number function to the calendar module of the stdlib application. The implementation conforms to the ISO 8601 standard. The new feature has been implemented tested and documented (thanks to Imre Horvath).

Own Id: OTP-9087

Improvements and New Features

Implement the 'MAY' clauses from RFC4648 regarding the pad character to make mime_decode() and mime_decode_to_string() functions more tolerant of badly padded base64. The RFC is quoted below for easy reference.

"RFC4648 Section 3.3 with reference to MIME decoding: Furthermore, such specifications MAY ignore the pad character, "=", treating it as non-alphabet data, if it is present before the end of the encoded data. If more than the allowed number of pad characters is found at the end of the string (e.g., a base 64 string terminated with "==="), the excess pad characters MAY also be ignored."

Own Id: OTP-9020

Supervisors will no longer save start parameters for temporary processes as they will not be restarted. In the case of simple_one_for_one workers such as ssl-connection processes this will substantial reduce the memory footprint of the supervisor.

Own Id: OTP-9064

When running escript it is now possible to add the -n flag and the escript will be compiled using +native.

Own Id: OTP-9076

STDLIB 1.17.2.1

Fixed Bugs and Malfunctions

Several type specifications for standard libraries were wrong in the R14B01 release. This is now corrected. The corrections concern types in re,io,filename and the module erlang itself.

Own Id: OTP-9008

STDLIB 1.17.2

Fixed Bugs and Malfunctions

When several clients accessed a Dets table simultaneously, one of them calling dets:insert_new/2, the Dets server could crash. Alternatively, under the same conditions, ok was sometimes returned instead of true. (Thanks to John Hughes.)

Own Id: OTP-8856

When several clients accessed a Dets table simultaneously, inserted or updated objects were sometimes lost due to the Dets file being truncated. (Thanks to John Hughes.)

Own Id: OTP-8898

When several clients accessed a Dets table simultaneously, modifications of the Dets server's internal state were sometimes thrown away. The symptoms are diverse: error with reason bad_object; inserted objects not returned by lookup(); et cetera. (Thanks to John Hughes.)

Own Id: OTP-8899

If a Dets table was closed after calling bchunk/2, match/1,3, match_object/1,3, or select/1,3 and then opened again, a subsequent call using the returned continuation would normally return a reply. This bug has fixed; now the call fails with reason badarg.

Own Id: OTP-8903

Cover did not collect coverage data for files such as Yecc parses containing include directives. The bug has been fixed by modifying epp, the Erlang Code Preprocessor.

Own Id: OTP-8911

If a Dets table with fewer slots than keys was opened and then closed after just a lookup, the contents were no longer well-formed. This bug has been fixed. (Thanks to Matthew Evans.)

Own Id: OTP-8923

In a supervisor, when it terminates a child, if that child happens to have exited fractionally early, with normal, the supervisor reports this as an error. This should not be reported as an error.

* POTENTIAL INCOMPATIBILITY *

Own Id: OTP-8938 Aux Id: seq11615

Improvements and New Features

The documentation filelib:wildcard/1,2 now describes the character set syntax for wildcards.

Own Id: OTP-8879 Aux Id: seq11683

Buffer overflows have been prevented in erlc, dialyzer, typer, run_test, heart, escript, and erlexec.

(Thanks to Michael Santos.)

Own Id: OTP-8892

Using a float for the number of copies for string:copies/2 resulted in an infinite loop. Now it will fail with an exception instead. (Thanks to Michael Santos.)

Own Id: OTP-8915

New ETS option compressed, to enable a more compact storage format at the expence of heavier table operations. For test and evaluation, erl +ec can be used to force compression on all ETS tables.

Own Id: OTP-8922 Aux Id: seq11658

The default maximum number of slots of a Dets table has been changed as to be equal to the maximum number of slots. (Thanks to Richard Carlsson.)

Own Id: OTP-8959

STDLIB 1.17.1

Fixed Bugs and Malfunctions

reference() has been substituted for ref() in the documentation.

Own Id: OTP-8733

Improvements and New Features

The ms_transform now warns if the fun head shadows surrounding variables (just like the warnings you would get for an ordinary fun in the same context).

Own Id: OTP-6759

ets:select_reverse/{1,2,3} are now documented.

Own Id: OTP-7863

Large parts of the ethread library have been rewritten. The ethread library is an Erlang runtime system internal, portable thread library used by the runtime system itself.

Most notable improvement is a reader optimized rwlock implementation which dramatically improve the performance of read-lock/read-unlock operations on multi processor systems by avoiding ping-ponging of the rwlock cache lines. The reader optimized rwlock implementation is used by miscellaneous rwlocks in the runtime system that are known to be read-locked frequently, and can be enabled on ETS tables by passing the {read_concurrency, true} option upon table creation. See the documentation of ets:new/2 for more information. The reader optimized rwlock implementation can be fine tuned when starting the runtime system. For more information, see the documentation of the +rg command line argument of erl.

There is also a new implementation of rwlocks that is not optimized for readers. Both implementations interleaves readers and writers during contention as opposed to, e.g., the NPTL (Linux) pthread rwlock implementation which use either a reader or writer preferred strategy. The reader/writer preferred strategy is problematic since it starves threads doing the non-preferred operation.

The new rwlock implementations in general performs better in ERTS than common pthread implementations. However, in some extremely heavily contended cases this is not the case. Such heavy contention can more or less only appear on ETS tables. This when multiple processes do very large amounts of write locked operations simultaneously on the same table. Such use of ETS is bad regardless of rwlock implementation, will never scale, and is something we strongly advise against.

The new rwlock implementations depend on atomic operations. If no native atomic implementation is found, a fallback solution will be used. Using the fallback implies a performance degradation. That is, it is more important now than before to build OTP with a native atomic implementation.

The ethread library contains native atomic implementations for, x86 (32 and 64 bit), powerpc (32 bit), sparc V9 (32 and 64 bit), and tilera (32 bit). On other hardware gcc's builtin support for atomic memory access will be used if such exists. If no such support is found, configure will warn about no atomic implementation available.

The ethread library can now also use the libatomic_ops library for atomic memory accesses. This makes it possible for the Erlang runtime system to utilize optimized native atomic operations on more platforms than before. If configure warns about no atomic implementation available, try using the libatomic_ops library. Use the --with-libatomic_ops=PATH configure command line argument when specifying where the libatomic_ops installation is located. The libatomic_ops library can be downloaded from: http://www.hpl.hp.com/research/linux/atomic_ops/

The changed API of the ethread library has also caused modifications in the Erlang runtime system. Preparations for the to come "delayed deallocation" feature has also been done since it depends on the ethread library.

Note: When building for x86, the ethread library will now use instructions that first appeared on the pentium 4 processor. If you want the runtime system to be compatible with older processors (back to 486) you need to pass the --enable-ethread-pre-pentium4-compatibility configure command line argument when configuring the system.

Own Id: OTP-8544

Some Built In Functions (BIFs) from the module erlang was never made autoimported for backward compatibility reasons. As local functions now override autoimports, new autoimports is no longer a problem, why the following BIFs are finally made autoimported: monitor/2, monitor/3, demonitor/2, demonitor/3, error/1, error/2, integer_to_list/2, list_to_integer/2.

Own Id: OTP-8763

STDLIB 1.17

Fixed Bugs and Malfunctions

The Erlang code preprocessor (epp) sent extra messages on the form {eof,Location} to the client when parsing the file attribute. This bug, introduced in R11B, has been fixed.

Own Id: OTP-8470

The abstract type 'fun' could not be printed by the Erlang pretty printer (erl_pp). This bug has been fixed.

Own Id: OTP-8473

The function erl_scan:reserved_word/1 no longer returns true when given the word spec. This bug was introduced in STDLIB-1.15.3 (R12B-3).

Own Id: OTP-8567

The documentation of lists:keysort/2 states that the sort is stable.

Own Id: OTP-8628 Aux Id: seq11576

The shell's line editing has been improved to more resemble the behaviour of readline and other shells. (Thanks to Dave Peticolas)

Own Id: OTP-8635

The Erlang code preprocessor (epp) did not correctly handle premature end-of-input when defining macros. This bug, introduced in STDLIB 1.16, has been fixed.

Own Id: OTP-8665 Aux Id: OTP-7810

Improvements and New Features

The module binary from EEP31 (and EEP9) is implemented.

Own Id: OTP-8217

The erlang pretty printer (erl_pp) no longer quotes atoms in types.

Own Id: OTP-8501

The Erlang code preprocessor (epp) now considers records with no fields as typed.

Own Id: OTP-8503

Added function zip:foldl/3 to iterate over zip archives.

Added functions to create and extract escripts. See escript:create/2 and escript:extract/2.

The undocumented function escript:foldl/3 has been removed. The same functionality can be achieved with the more flexible functions escript:extract/2 and zip:foldl/3.

Record fields has been annotated with type info. Source files as been adapted to fit within 80 chars and trailing whitespace has been removed.

Own Id: OTP-8521

The Erlang parser no longer duplicates the singleton type undefined in the type of record fields without initial value.

Own Id: OTP-8522

A regular expression with many levels of parenthesis could cause a buffer overflow. That has been corrected. (Thanks to Michael Santos.)

Own Id: OTP-8539

When defining macros the closing right parenthesis before the dot is now mandatory.

* POTENTIAL INCOMPATIBILITY *

Own Id: OTP-8562

Some properties of a compiled re pattern are defined to allow for guard tests.

Own Id: OTP-8577

Local and imported functions now override the auto-imported BIFs when the names clash. The pre R14 behaviour was that auto-imported BIFs would override local functions. To avoid that old programs change behaviour, the following will generate an error:

Doing a call without explicit module name to a local function having a name clashing with the name of an auto-imported BIF that was present (and auto-imported) before OTP R14A

Explicitly importing a function having a name clashing with the name of an autoimported BIF that was present (and autoimported) before OTP R14A

Using any form of the old compiler directive nowarn_bif_clash

If the BIF was added or auto-imported in OTP R14A or later, overriding it with an import or a local function will only result in a warning,

To resolve clashes, you can either use the explicit module name erlang to call the BIF, or you can remove the auto-import of that specific BIF by using the new compiler directive -compile({no_auto_import,[F/A]})., which makes all calls to the local or imported function without explicit module name pass without warnings or errors.

The change makes it possible to add auto-imported BIFs without breaking or silently changing old code in the future. However some current code ingeniously utilizing the old behaviour or the nowarn_bif_clash compiler directive, might need changing to be accepted by the compiler.

* POTENTIAL INCOMPATIBILITY *

Own Id: OTP-8579

The undocumented, unsupport, and deprecated function lists:flat_length/1 has been removed.

Own Id: OTP-8584

A bug in re that could cause certain regular expression matches never to terminate is corrected. (Thanks to Michael Santos and Gordon Guthrie.)

Own Id: OTP-8589

Nested records can now be accessed without parenthesis. See the Reference Manual for examples. (Thanks to YAMASHINA Hio and Tuncer Ayaz.)

Own Id: OTP-8597

receive statements that can only read out a newly created reference are now specially optimized so that it will execute in constant time regardless of the number of messages in the receive queue for the process. That optimization will benefit calls to gen_server:call(). (See gen:do_call/4 for an example of a receive statement that will be optimized.)

Own Id: OTP-8623

The beam_lib:cmp/2 function now compares BEAM files in stricter way. The BEAM files will be considered different if there are any changes except in the compilation information ("CInf") chunk. beam_lib:cmp/2 used to ignore differences in the debug information (significant for Dialyzer) and other chunks that did not directly change the run-time behavior.

Own Id: OTP-8625

When a gen_server, gen_fsm process, or gen_event terminates abnormally, sometimes the text representation of the process state can occupy many lines of the error log, depending on the definition of the state term. A mechanism to trim out parts of the state from the log has been added (using a format_status/2 callback). See the documentation.

Own Id: OTP-8630

Calling sys:get_status() for processes that have globally registered names that were not atoms would cause a crash. Corrected. (Thanks to Steve Vinoski.)

Own Id: OTP-8656

The Erlang scanner has been augmented with two new tokens: .. and ....

Own Id: OTP-8657

Expressions evaluating to integers can now be used in types and function specifications where hitherto only integers were allowed ("Erlang_Integer").

Own Id: OTP-8664

The compiler optimizes record operations better.

Own Id: OTP-8668

The recently added BIFs erlang:min/2, erlang:max/2 and erlang:port_command/3 are now auto-imported (as they were originally intended to be). Due to the recent compiler change (OTP-8579), the only impact on old code defining it's own min/2, max/2 or port_command/3 functions will be a warning, the local functions will still be used. The warning can be removed by using -compile({no_auto_import,[min/2,max/2,port_command/3]}). in the source file.

* POTENTIAL INCOMPATIBILITY *

Own Id: OTP-8669 Aux Id: OTP-8579

Now, binary_to_term/2 is auto-imported. This will cause a compile warning if and only if a module has got a local function with that name.

* POTENTIAL INCOMPATIBILITY *

Own Id: OTP-8671

The predefined builtin type tid() has been removed. Instead, ets:tid() should be used.

* POTENTIAL INCOMPATIBILITY *

Own Id: OTP-8687

STDLIB 1.16.5

Fixed Bugs and Malfunctions

Because of a race condition, using filelib:ensure_dir/1 from multiple processes to create the same path or parts of the same directory structure, filelib:ensure_dir/1 could return a meaningless {error,eexist}. That race condition has been eliminated, and {error,eexist} will now be returned only if there exists a regular file, device file, or some other non-directory file with the same name. (Thanks to Tuncer Ayaz.)

Own Id: OTP-8389

A number of bugs concerning re and unicode are corrected:

re:compile no longer loses unicode option, which also fixes bug in re:split.

re:replace now handles unicode charlist replacement argument

re:replace now handles unicode RE charlist argument correctly

re:replace now handles binary unicode output correctly when nothing is replaced.

Most code, testcases and error isolation done by Rory Byrne.

Own Id: OTP-8394

The loading of native code was not properly atomic in the SMP emulator, which could cause crashes. Also a per-MFA information table for the native code has now been protected with a lock since it turns that it could be accessed concurrently in the SMP emulator. (Thanks to Mikael Pettersson.)

Own Id: OTP-8397

user.erl (used in oldshell) is updated to handle unicode in prompt strings (io:get_line/{1,2}). io_lib is also updated to format prompts with the 't' modifier (i.e. ~ts instead of ~s).

Own Id: OTP-8418 Aux Id: OTP-8393

The re module: A regular expression with an option change at the start of a pattern that had top-level alternatives could cause overwriting and/or a crash. (Thanks to Michael Santos.)

Own Id: OTP-8438

Improvements and New Features

The ability for the gen_server and gen_fsm callback modules to format their own state for display under the sys:get_status/1,2 calls has been restored and documented. (Thanks to Steve Vinoski.)

Own Id: OTP-8324

c:nc/{1,2} used to assume that the beam file was created in the same directory as the source code and failed to load the code if it was not. Corrected to look for the beam file in the current directory or in the directory specified by the {outdir,Dir} option. (Thanks to Alex Suraci.)

Own Id: OTP-8337

The documentation is now possible to build in an open source environment after a number of bugs are fixed and some features are added in the documentation build process.

- The arity calculation is updated.

- The module prefix used in the function names for bif's are removed in the generated links so the links will look like "http://www.erlang.org/doc/man/erlang.html#append_element-2" instead of "http://www.erlang.org/doc/man/erlang.html#erlang:append_element-2".

- Enhanced the menu positioning in the html documentation when a new page is loaded.

- A number of corrections in the generation of man pages (thanks to Sergei Golovan)

- The legal notice is taken from the xml book file so OTP's build process can be used for non OTP applications.

Own Id: OTP-8343

Shell tab completion now works for quoted module and function names. (Thanks to Ulf Wiger.)

Own Id: OTP-8383

Explicit top directories in archive files are now optional.

For example, if an archive (app-vsn.ez) just contains an app-vsn/ebin/mod.beam file, the file info for the app-vsn and app-vsn/ebin directories are faked using the file info from the archive file as origin. The virtual direcories can also be listed. For short, the top directories are virtual if they does not exist.

Own Id: OTP-8387

Macros overloading has been implemented. (Thanks to Christopher Faulet.)

Own Id: OTP-8388

The new function shell:prompt_func/1 and the new application configuration parameter shell_prompt_func can be used for customizing the Erlang shell prompt.

Own Id: OTP-8393

Improved handling of typed records in escripts

Own Id: OTP-8434

Added supervisor:count_children/1 to count the number of children being managed without the memory impact of which_children/1. (Thanks to Jay Nelson.)

Own Id: OTP-8436

STDLIB 1.16.4

Improvements and New Features

The documentation is now built with open source tools (xsltproc and fop) that exists on most platforms. One visible change is that the frames are removed.

Own Id: OTP-8201

[escript] The restriction that the first line in escripts must begin with #! has been removed.

[escript] Some command line options to the escript executable has now been documented. For example you can run an escript in the debugger by just adding a command line option.

[escript] The documentation of the escript header syntax has been clarified. For example the header is optional. This means that it is possible to directly "execute" .erl, .beam and.zip files.

Own Id: OTP-8215

Optimized array:from_orddict/1, it is now faster and uses less memory if the orddict was sparse.

Changed array:reset/2, it will now never expand the array which it could before for non fixed arrays. See the documentation.

Own Id: OTP-8216

The Erlang Pretty Printer (erl_pp) now puts the leading [ of list comprehensions as well as the leading << of bit string comprehensions on a separate line in order to expose the Cover counter of the template.

Own Id: OTP-8227

The extension ".xrl" used for Leex input files is now recognized by the compiler.

Own Id: OTP-8232

Some clarifications have been made in the documentation regarding gen_server, gen_fsm, and gen_event behavior when handling 'EXIT' messages from the parent process. For more information see the gen_server, gen_fsm, and gen_event documentation.

Own Id: OTP-8255 Aux Id: seq11419

The -on_load() directive can be used to run a function when a module is loaded. It is documented in the section about code loading in the Reference Manual.

Own Id: OTP-8295

STDLIB 1.16.3.1

Fixed Bugs and Malfunctions

An erroneous type spec for gen:start/6 caused dialyzer to erroneously issue warnings when {spawn_opt, SpawnOptionList} was passed in the option list to the gen_server and gen_fsm start functions.

Own Id: OTP-8068 Aux Id: seq11323, seq11314

STDLIB 1.16.3

Fixed Bugs and Malfunctions

The linter used to crash on invalid -opaque declarations.

Own Id: OTP-8051

Bugs in digraph:add_edge/5 and digraph:del_path/3 have been fixed. (Thanks to Crystal Din.)

Own Id: OTP-8066

When trying to insert objects with dets:insert_new() into a Dets table of type duplicate_bag, already existing objects would sometimes be duplicated. This bug has been fixed. (Thanks to Crystal Din.)

Own Id: OTP-8070

Running erlc in a very deep directory (with a path length of more 256 or more characters) would cause the emulator to crash in a call to list_to_atom/1. (Thanks to Chris Newcombe.)

Own Id: OTP-8124

A few minor bugs have been fixed in the Erlang Code Preprocessor (epp).

Own Id: OTP-8130

A bug in The Erlang Meta Interpreter (erl_eval) has been fixed: exceptions generated in the template of bit string comprehensions were not handled properly. (Thanks to Ulf Wiger.)

Own Id: OTP-8133

Improvements and New Features

Option {capture,none} was missing in documentation for re:run/3.

Own Id: OTP-8113

When erl_scan:tokens() returns an error tuple {error, ErrorInfo, EndLocation}, the list LeftOverChars is the remaining characters of the input data, starting from EndLocation. It used to be the empty list.

* POTENTIAL INCOMPATIBILITY *

Own Id: OTP-8129

The Erlang Meta Interpreter (erl_eval) has been somewhat optimized when it comes to interpreting receive-expressions. (Thanks to Richard Carlsson.)

Own Id: OTP-8139

The Erlang Pretty Printer (erl_pp) has been modified as to handle types.

Own Id: OTP-8150

STDLIB 1.16.2

Fixed Bugs and Malfunctions

The text of tokens returned by the Erlang scanner (erl_scan) was sometimes empty when the text option was given and StartLocation was a line. This bug has been fixed.

Own Id: OTP-7965

The documentation for base64:decode/1 has been updated to point out that it strips whitespace.

base64:decode/1 and base64:mime_decode/1 would sometimes fail instead of stripping away non-base64 characters.

Own Id: OTP-7984

Two types in the gen module were corrected.

Own Id: OTP-8029 Aux Id: seq11296

array:from_orddict([]) and array:from_list([]) would construct fixed arrays instead of extendible arrays.

Own Id: OTP-8033

Improvements and New Features

Interpreted escripts are now tail recursive.

The function erl_eval:expr/5 has been introduced.

Own Id: OTP-7933

gen_server:call/2,3 will be somewhat faster if the calling process has a many messages in its message queue.

Own Id: OTP-7979

Random now supports seed with arity one, random:seed/1, which takes a three-tuple.

Own Id: OTP-8019

The regexp module now recognizes the escape sequences \xXY and \x{X...}.

Own Id: OTP-8024

STDLIB 1.16.1

Fixed Bugs and Malfunctions

The documentation of dets:open_file/1 now states that the file is repaired if it has not been properly closed. (Thanks to Ulf Wiger.)

Own Id: OTP-7895

Improvements and New Features

The Erlang scanner no longer returns the text of tokens when the start location is a pair of a line and column unless the new option text is supplied (incompatibility with R13A).

There are new functions to access the attributes of tokens: attributes_info/1,2 and set_attribute/3.

* POTENTIAL INCOMPATIBILITY *

Own Id: OTP-7892 Aux Id: OTP-7810

Several glitches and performance issues in the Unicode and I/O-system implementation of R13A have been corrected.

Own Id: OTP-7896 Aux Id: OTP-7648 OTP-7887

The type spec of filelib:wildcard/2 has been corrected.

Own Id: OTP-7915

New functions: gb_sets:is_disjoint/2, ordsets:is_disjoint/2, and gb_sets:is_disjoint/2.

Own Id: OTP-7947

The function gb_trees:map/2 which was added in R13A is now documented.

Own Id: OTP-7948

STDLIB 1.16

Fixed Bugs and Malfunctions

Fixed a minor race conditions in gen_server:start*: if one of these functions returned {error,Reason} or ignore, the name could still be registered (either locally or in global).

A process started by proc_lib in some cases depended on its process dictionary not to be erased, and would crash when terminating abnormally and not generate a proper crash report. This has been corrected (but the initial call will not be shown in the error report if the process dictionary has been erased). NOTE: There is no longer any need to erase the process dictionary for memory conservation reasons, since the actual call arguments are no longer saved in the process dictionary.

Own Id: OTP-7669

The Erlang preprocessor used wrong line number when stringifying macro arguments. (Thanks to John Hughes.)

Own Id: OTP-7702

A bug in the qlc module has been fixed: merge join sometimes failed to return all answers. (Thanks to Bernard Duggan.)

Own Id: OTP-7714

Improvements and New Features

A new option, key_equality, has been added to qlc:table/2. This option makes it possible for qlc to better handle tables that use ==/2 when comparing keys for equality (examples of such tables are ordered ETS tables and gb_table in qlc(3)).

Own Id: OTP-6674

The functions lists:seq/1,2 return the empty list in a few cases when they used to generate an exception, for example lists:seq(1, 0). See lists(3) for details. (Thanks to Richard O'Keefe.)

* POTENTIAL INCOMPATIBILITY *

Own Id: OTP-7230

The order of objects visited in select for ordered_set is now documented.

Own Id: OTP-7339

It is now possible to debug code in escripts and archives.

Own Id: OTP-7626

Support for Unicode is implemented as described in EEP10. Formatting and reading of unicode data both from terminals and files is supported by the io and io_lib modules. Files can be opened in modes with automatic translation to and from different unicode formats. The module 'unicode' contains functions for conversion between external and internal unicode formats and the re module has support for unicode data. There is also language syntax for specifying string and character data beyond the ISO-latin-1 range.

The interactive shell will support input and output of unicode characters when the terminal and operating system supports it.

Please see the EEP and the io/io_lib manual pages as well as the stdlib users guide for details.

I/O-protocol incompatibilities:

The ioprotocol between io_Server and client is updated to handle protocol data in unicode formats. The updated protocol is now documented. The specification resides in the stdlib _users manual, which is a new part of the manual.

io module incompatibilities:

The io:putchars, io:get_chars and io:get_line all handle and return unicode data. In the case where binaries can be provided (as to io:put_chars), they shall be encoded in UTF-8. When binaries are returned (as by io:get_line/get_chars when the io_server is set in _binary mode) the returned data is also always encoded as UTF-8. The file module however still returns byte-oriented data, why file:read can be used instead of io:get_chars to read binary data in ISO-latin-1.

io_lib module incompatibilities:

io_lib:format can, given new format directives (i.e "~ts" and "~tc"), return lists containing integers larger than 255.

* POTENTIAL INCOMPATIBILITY *

Own Id: OTP-7648 Aux Id: OTP-7580 OTP-7514 OTP-7494 OTP-7443 OTP-7181 EEP10 EEP11

The function pool:attach/1 now returns already_attached if the node is already attached, rather than allready_attached (sic!). (Thanks to Edwin Fine.)

Own Id: OTP-7653 Aux Id: OTP-7603

Preprocessor directives are now allowed in escripts. This means that for example macros may be used in escripts.

Own Id: OTP-7662

When a process started with proc_lib, gen_server, or gen_fsm exits with reason {shutdown,Term}, a crash report will no longer be generated (to allow a clean shutdown, but still provide additional information to process that are linked to the terminating process).

Own Id: OTP-7740 Aux Id: seq10847

A new BIF, lists:keyfind/3, has been added. It works like lists:keysearch/3 except that it does not wrap the returned tuple in a value tuple in case of success. (Thanks to James Hague for suggesting this function.)

Own Id: OTP-7752

lists:suffix(Suffix, List) used to have a a complexity of length(Suffix)*length(List) (which could become quite slow for some inputs). It has now been re-implemented so that its complexity is length(Suffix)+length(List). (Thanks to Richard O'Keefe for the new implementation.)

Own Id: OTP-7797

The Erlang scanner has been augmented as to return white spaces, comments, and exact location of tokens. The functions string/3, tokens/4, and token_info/1,2 are new. See erl_scan(3) for details.

tokens/3,4 have been modified as to return a list of tokens instead of an error when eof is encountered before the dot.

Own Id: OTP-7810

filelib:fold_files/5 now uses the re module instead of the regexp module for regular expression matching. In practice, this change will not be a problem for most regular expressions used for filelib:fold_files/5. (The major difference in regular expression is that parenthesis and curly brackets is treated as literal characters by regexp but as special characters by re; fortunately, those characters are rarely used in filenames.)

* POTENTIAL INCOMPATIBILITY *

Own Id: OTP-7819

digraph:new(Type) will now cause a badarg exception if Type is not a valid type. Similarly, digraph_utils:subgraph/2,3 will now cause a badarg if the arguments are invalid. (Those functions used to return error tuples if something was wrong.)

* POTENTIAL INCOMPATIBILITY *

Own Id: OTP-7824

The argument passed to random:uniform/1 must now be an integer (as stated in the documentation). In previous releases, a floating point number was also allowed.

* POTENTIAL INCOMPATIBILITY *

Own Id: OTP-7827

The copyright notices have been updated.

Own Id: OTP-7851

A few missing match spec functions was added to dbg:fun2ms; exception_trace/0 and trace/2,3.

There is a new function queue:member/2.

A bug in io_lib:fread that made it accidentally concatenate fields separated by newline has been corrected. Reported and analyzed by Matthew Palmer to erlang-patches.

Own Id: OTP-7865

STDLIB 1.15.5

Fixed Bugs and Malfunctions

A bug in the qlc module has been fixed: when merge joining two query handles the temporary file used for equivalence classes was not truncated properly which could result in poor performance.

Own Id: OTP-7552

The characters 16#C0 and 16#E0 ("A" and "a" with grave accent), were not properly converted by the string:to_lower/1 and string:to_upper/1 functions. (Thanks to Richard O'Keefe.)

Own Id: OTP-7589

The function pool:attach/1 now returns already_attached if the node is already attached, rather than allready_attached (sic!). (Thanks to Edwin Fine.)

* POTENTIAL INCOMPATIBILITY *

Own Id: OTP-7603

The documentation for io:get_line/1,2 now mentions that the return value can also be {error,Reason}.

Own Id: OTP-7604 Aux Id: seq11063

Improvements and New Features

The split function is now added to the re library. Exceptions and errors from both run, replace and split are made more consistent.

Own Id: OTP-7514 Aux Id: OTP-7494

Processes spawned using proc_lib (including gen_server and other library modules that use proc_lib) no longer keep the entire argument list for the initial call, but only the arity.

Also, if proc_lib:spawn/1 is used to spawn a fun, the actual fun is not kept, but only module, function name, and arity of the function that implements the fun.

The reason for the change is that keeping the initial fun (or a fun in an argument list), would prevent upgrading the code for the module. A secondary reason is that keeping the fun and function arguments could waste a significant amount of memory.

The drawback with the change is that the crash reports will provide less precise information about the initial call (only Module:Function/Arity instead of Module:Function(Arguments)). The function proc_lib:initial_call/1 still returns a list, but each argument has been replaced with a dummy atom.

Own Id: OTP-7531 Aux Id: seq11036

There is now experimental support for loading of code from archive files. See the documentation of code, init, erl_prim_loaderand escript for more info.

The error handling of escripts has been improved.

An escript may now set explicit arguments to the emulator, such as -smp enabled.

An escript may now contain a precompiled beam file.

An escript may now contain an archive file containing one or more applications (experimental).

The internal module code_aux has been removed.

Own Id: OTP-7548 Aux Id: otp-6622

Enabled explicit control of which types of files that should be compressed in a ZIP archive.

Own Id: OTP-7549 Aux Id: otp-6622

In the job control mode, the "s" and "r" commands now take an optional argument to specify which shell to start. (Thanks to Robert Virding.)

Own Id: OTP-7617

STDLIB 1.15.4

Fixed Bugs and Malfunctions

A bug in the calendar module could cause calendar:local_time_to_universal_time_dst/1 to return duplicate identical values for local times in timezones without DST. Multiple values should only be returned when a local time is within the hour occurring twice due to shift from DST to non-DST, and certainly only in timezones with DST. The correct behaviour is now implemented.

Own Id: OTP-7344 Aux Id: seq10960

The documentation of (d)ets:init_table() has been corrected. (Thanks to Paul Mineiro.)

Own Id: OTP-7413

The soft upper limit of 60 on the number of non-white characters on a line, which was introduced in R12B-0 for the control sequences p and P of the functions io:fwrite/2,3 and io_lib:fwrite/2, has been removed. This means that terms whose printed representation fits on a line will have no NEWLINEs. The Erlang shell still uses the 60 character limit, though.

Own Id: OTP-7421 Aux Id: OTP-6708

Some debug code has been removed from Dets.

Own Id: OTP-7424

The documentation of dets:match_delete/2 has been corrected. (Thanks to Paul Mineiro.)

Own Id: OTP-7445

Corrections of digraph(3). (Thanks to Vlad Dumitrescu.)

Own Id: OTP-7492

For the process that an escript runs in, the trap_exit process flag is now false instead of true (as in previous releases). Scripts that depend on the previous (counter-intuitive) behaviour might not work. (Thanks to Bengt Kleberg.)

* POTENTIAL INCOMPATIBILITY *

Own Id: OTP-7517

Improvements and New Features

The documentation of lists:(u)sort/2 now states what is expected of an ordering function.

Own Id: OTP-7489

The re module is extended with repetitive matches (global option) and replacement function.

Own Id: OTP-7494 Aux Id: OTP-7181

The Erlang shell now displays a nicer error message when evaluating an undefined command. (Thanks to Richard Carlsson.)

Own Id: OTP-7495

STDLIB 1.15.3

Fixed Bugs and Malfunctions

zip:unzip to/from binary with empty directories did not work. (Thanks to Martin Dvorak.)

Own Id: OTP-7248

The documentation of the control sequence w of the io_lib module now states that floating point numbers are printed accurately.

Own Id: OTP-7324 Aux Id: OTP-7084

zip:unzip was not supporting a flavour of the zip format found in jar-files.

Own Id: OTP-7382 Aux Id: seq10970

Improvements and New Features

An experimental module "re" is added to the emulator which interfaces a publicly available regular expression library for Perl-like regular expressions (PCRE). The interface is purely experimental and will be subject to change.

The implementation is for reference and testing in connection to the relevant EEP.

Own Id: OTP-7181

STDLIB 1.15.2

Fixed Bugs and Malfunctions

When inserting many small objects, Dets sometimes crashed when reaching the maximum number of slots. (Thanks to Daniel Goertzen.)

Own Id: OTP-7146

Processes linked to the Erlang shell did not get an exit signal when the evaluator process was killed. This bug, introduced in R12B-0, has been fixed.

Own Id: OTP-7184 Aux Id: OTP-6554

Invalid arguments to ets:update_counter/3 were not handled correctly. A tuple position (Pos) less than 1 caused the element directly following the key to be updated (as if no position at all had been specified). All invalid values for Pos will now fail with badarg.

Own Id: OTP-7226

For certain terminals, io:columns/0 could return 0 instead of enotsup. That is now corrected.

Own Id: OTP-7229 Aux Id: seq10886

qlc:info() can now handle port identifiers, pids, references, and funs. (Thanks to Wojciech Kaczmare for reporting this bug.)

When evaluating the parent_fun messages sent to the process calling qlc:cursor() were sometimes erroneously consumed. This bug has been fixed.

Own Id: OTP-7232

erl_parse:abstract() can now handle bit strings.

Own Id: OTP-7234

Improvements and New Features

The queue module has been rewritten to make it easier to use. Suggestions and discussion from and with among others Lev Walkin, Anders Ramsell and Rober Virding in december 2007 on erlang-questions@erlang.org. It was also discussed to change the internal representation to contain length information which would speed up len/1 but that change has been postponed. Anyone interested may write an EEP and try to reach an acceptable compromise for queue overhead and thereby the speed of all other operations than len/1. The queue module is now optimized for fast and minimal garbage in/2 and out/1 and such. See the documentation.

New functions: is_queue/1, get/1, get_r/1, peek/1, peek_r/1, drop/1, drop_r/1 and liat/1. is_queue/1 is a new predicate, liat/1 is a correction of an old misspelling, and the others (get*, peek* and drop*) are new interface functions.

Own Id: OTP-7064

The functions io_lib:write/1,2 and io_lib:print/1,4 have been changed when it comes to writing floating point numbers. This change affects the control sequences p, P, w, and W of the io_lib module. (Thanks to Bob Ippolito for code contribution.)

Own Id: OTP-7084

Updated the documentation for erlang:function_exported/3 and io:format/2 functions to no longer state that those functions are kept mainly for backwards compatibility.

Own Id: OTP-7186

A new BIF ets:update_element/3. To update individual elements within an ets-tuple, without having to read, update and write back the entire tuple.

Own Id: OTP-7200

string:join/2 now accepts an empty list as first argument.

Own Id: OTP-7231 Aux Id: OTP-6671

qlc:info/1,2 accepts a new option, depth. The type SelectedObjects used in the description of qlc:table/2 has been augmented.

Own Id: OTP-7238

tuple_size/1 and byte_size/1 have been substituted for size/1 in the documentation.

Own Id: OTP-7244

STDLIB 1.15.1

Fixed Bugs and Malfunctions

Ets:select/3 in combination with ets:repair_continuation/2 and ordered_set data tables could result in function_clause although used as intended. This is now corrected. Thanks to Paul Mineiro for finding and isolating the bug!

Own Id: OTP-7025

The compiler warning for the deprecated function ftp:close/1 now mentions the correct replacement function.

The warning for the removed functions in the httpd_util module have been changed to say they have been removed, not merely deprecated. (Thanks to Fredrik Thulin.)

Own Id: OTP-7034 Aux Id: seq10825

In (Expr)#r{} (no fields are updated), Expr is no longer evaluated more than once. There is also a test that Expr is of the correct record type. (Thanks to Dominic Williams.)

Own Id: OTP-7078 Aux Id: OTP-4962

Documentation bugfixes and clarifications.

(Thanks to Joern (opendev@gmail.com), Matthias Lang, and Richard Carlsson.)

Own Id: OTP-7079

Duplicated objects were sometimes not deleted from the list of answers when a QLC table was traversed using a match specification. (Thanks to Dmitri Girenko.)

Own Id: OTP-7114

Improvements and New Features

The documentation has been updated so as to reflect the last updates of the Erlang shell as well as the minor modifications of the control sequence p of the io_lib module.

Superfluous empty lines have been removed from code examples and from Erlang shell examples.

Own Id: OTP-6944 Aux Id: OTP-6554, OTP-6911

tuple_size/1 and byte_size/1 have been substituted for size/1.

Own Id: OTP-7009

It is now possible to hibernate a gen_server/gen_event/gen_fsm. In gen_server and gen_fsm, hibernation is triggered by returning the atom 'hibernate' instead of a timeout value. In the gen_event case hibernation is triggered by a event handler returning a tuple with an extra element containing the atom 'hibernate'.

Own Id: OTP-7026 Aux Id: seq10817

Some undocumented debug functionality has been added to Dets.

Own Id: OTP-7066

The functions digraph_utils:is_tree/1, digraph_utils:is_arborescence/1, and digraph_utils:arborescence_root/1 are new.

Own Id: OTP-7081

The compiler could generate suboptimal code for record updates if the record update code consisted of multiple source code lines.

Own Id: OTP-7101

STDLIB 1.15

Fixed Bugs and Malfunctions

Bugs have been fixed in qlc:

Setting the lookup_fun option of qlc:table/2 to undefined could cause a crash.

If a QLC restricted some column of a table in such a way that a traversal using a match specification was possible and the QLC also compared the key column or some indexed column of the the table with a column of some other table, qlc always chose to traverse the table first, never considering lookup join. This has been changed so that lookup join is always preferred; if an initial traversal using the match specification is desired, the query needs to be rewritten introducing an extra QLC with the filter(s) restricting the column.

When trying to find candidates for match specifications and lookup, filters using variables from one generator only are ignored unless they are placed immediately after the generator and possibly other filters using variables from the same generator. In particular, filters joining two tables should not be placed between the generator and the filters using the generator only.

The call-back function TraverseFun used for implementing QLC tables is allowed to return a term other than a list since STDLIB 1.14 (OTP-5195). However, when the returned term was a fun qlc often tried to call the fun instead of returning it.

A few minor optimizations have been implemented as well.

Own Id: OTP-6673

A bug concerning the use of parameterized modules from the shell has been fixed.

Own Id: OTP-6785

A bug regarding the size expression of the bit syntax has been fixed in the erl_eval module.

Own Id: OTP-6787

The log_mf_h event handler didn't close the index file when it was done reading it causing a file descriptor leak.

Own Id: OTP-6800

Definitions for the filename() and dirname() types have been added to the documentation for the filelib module.

Own Id: OTP-6870

file:write_file/3, file:write/2 and file:read/2 could crash (contrary to documentation) for odd enough file system problems, e.g write to full file system. This bug has now been corrected.

In this process the file module has been rewritten to produce better error codes. Posix error codes now originate from the OS file system calls or are generated only for very similar causes (for example 'enomem' is generated if a memory allocation fails, and 'einval' is generated if the file handle in Erlang is a file handle but currently invalid).

More Erlang-ish error codes are now generated. For example {error,badarg} is now returned from file:close/1 if the argument is not of a file handle type. See file(3).

The possibility to write a single byte using file:write/2 instead of a list or binary of one byte, contradictory to the documentation, has been removed.

* POTENTIAL INCOMPATIBILITY *

Own Id: OTP-6967 Aux Id: OTP-6597 OTP-6291

A bug concerning the evaluation of the ++/2 operator has been fixed in erl_eval. (Thanks to Matthew Dempsky.)

Own Id: OTP-6977

Improvements and New Features

The behaviour of the internal functions gen:call/3,4 has been changed slightly in the rare case that when the caller was linked to the called server, and the server crashed during the call; its exit signal was consumed by the gen:call/3,4 code and converted to an exit exception. This exit signal is no longer consumed.

To even notice this change, 1) the calling process has to be linked to the called server.

the call must not be remote by name that is it must be local or remote by pid, local by name or global by name.

the calling process has to have set process_flag(trap_exit, true).

the server has to crash during the call.

the calling process has to be sensitive to getting previously consumed {'EXIT',Pid,Reason} messages in its message queue.

The old behaviour was once the only way for a client to notice if the server died, but has since erlang:monitor(process, {Name,Node}) was introduced and used in gen:call been regarded as an undesired behaviour if not a bug.

The affected user APIs are: gen_server:call/2,3, gen_fsm:sync_send_event/2,3, gen_fsm:sync_send_all_state_event/2,3, gen_event:_, sys:_ and maybe a few others that hardly will be noticed.

* POTENTIAL INCOMPATIBILITY *

Own Id: OTP-3954 Aux Id: Seq 4538

When an exception occurs the Erlang shell now displays the class, the reason, and the stacktrace in a clearer way (rather than dumping the raw EXIT tuples as before). proc_lib:format/1 displays the exception of crash reports in the same clearer way.

The new shell command catch_exception and the new application configuration parameter shell_catch_exception can be used for catching exceptions that would normally exit the Erlang shell.

Own Id: OTP-6554 Aux Id: OTP-6289

The function string:join/2 joins strings in a list with a separator. Example: 'string:join(["a", "b", "c"], ", ") gives "a, b, c"'

Own Id: OTP-6671

The control sequence P of the Format argument of the functions io:fwrite/2,3 and io_lib:fwrite/2 now inserts fewer line breaks when printing tuples and lists. A soft upper limit of 60 on the number of non-white characters on a line has been introduced.

Own Id: OTP-6708

The new module array provides a fast functional array implementation.

Own Id: OTP-6733

Functions that have long been deprecated have now been removed from the following modules: dict, erl_eval, erl_pp, io, io_lib, lists, orddict, ordsets, sets, and string.

The undocumented function lists:zf/3 has also been removed (use a list comprehension or lists:zf/2 instead).

* POTENTIAL INCOMPATIBILITY *

Own Id: OTP-6845

Minor documentation corrections for file:pread/2 and file:pread/3.

Own Id: OTP-6853

Contract directives for modules in Kernel and STDLIB.

Own Id: OTP-6895

The ets:fixtable/2 function, which has been deprecated for several releases, has been removed.

The ets:info/1 function has been reimplemented as a BIF, which guarantees that information returned is consistent.

The ets:info/2 function now fails with reason badarg if the second argument is invalid. (Dialyzer can be used to find buggy code where the second argument is misspelled.)

* POTENTIAL INCOMPATIBILITY *

Own Id: OTP-6906

The Erlang pretty printer erl_pp now inserts more newlines in order to facilitate line coverage analysis by Cover. (Thanks to Thomas Arts.)

Own Id: OTP-6911

The documentation for ets:safe_fixtable/2, ets:foldl/3, and ets:foldr/3 is now clearer about what will happen if objects are inserted during table traversals.

Own Id: OTP-6928 Aux Id: seq10779

It is now possible to extract files in tar files directly into binaries. It is also possible to add files to tar files directly from binaries.

Own Id: OTP-6943

The functions keystore/4 and keytake/3 are new in the lists module.

Own Id: OTP-6953

The new qlc option tmpdir_usage can be used for outputting messages onto the error logger when a temporary file is about to be created, or to prohibit the usage of temporary files altogether.

Own Id: OTP-6964

STDLIB 1.14.5.3

Improvements and New Features

The allowed syntax for -type() and -spec() was updated.

Own Id: OTP-6861 Aux Id: OTP-6834

STDLIB 1.14.5.2

Improvements and New Features

The compiler will for forward compatibility ignore the -type() and -spec() attributes that will be introduced in the R12B release.

Own Id: OTP-6834

STDLIB 1.14.5.1

Fixed Bugs and Malfunctions

The log_mf_h event handler didn't close the index file when it was done reading it causing a file descriptor leak.

Own Id: OTP-6800

Improvements and New Features

The dict:size/1 and orddict:size/1 functions have been documented.

Own Id: OTP-6818

STDLIB 1.14.5

Fixed Bugs and Malfunctions

Bugs have been fixed in Dets concerning comparison (==) and matching (=:=).

The STDLIB manual pages have been updated as to more carefully state when terms are matched and when they are compared.

Own Id: OTP-4738 Aux Id: OTP-4685

The shell has been updated to fix the following flaws: Shell process exit left you with an unresponsive initial shell if not using oldshell. Starting a restricted shell with a nonexisting callback module resulted in a shell where no commands could be used, not even init:stop/0. Fun's could not be used as parameters to local shell functions (in shell_default or user_default) when restricted_shell was active.

Own Id: OTP-6537

A bug in QLC's parse transform has been fixed.

Own Id: OTP-6590

A bug concerning lists:sort/1 and lists:keysort/2 and a mix of floating point numbers and integers has been fixed.

Own Id: OTP-6606

When calling erlang:garbage_collect/0 in the Erlang shell not only the evaluator process (the one returned by calling self/0 in the Erlang shell) is garbage collected, but also the process holding the history list.

Own Id: OTP-6659

Functions of the beam_lib module that used to catch exceptions and return a tuple {'EXIT',Reason} now exit with the reason Reason.

Own Id: OTP-6711

The erl_eval module now calls the non-local function handler whenever an operator is evaluated (exceptions are andalso, orelse, and catch). The non-local function handler is now also called when the function or operator occurs in a guard test (such calls used to be ignored).

These changes affect the Erlang shell when running in restricted mode: the callback function non_local_allowed/3 is now called for operators such as '!'/2. This means that non_local_allowed/3 may need to be changed as to let operators through. Note that erlang:'!'/2 as well as erlang:send/2,3 have to be restricted in order to stop message passing in the shell.

* POTENTIAL INCOMPATIBILITY *

Own Id: OTP-6714 Aux Id: seq10374

Improvements and New Features

The new compiler option warn_obsolete_guard can be used for turning on warnings for calls to old type testing BIFs.

Own Id: OTP-6585

For scripts written using escript, there is a new function escript:script_name/0, which can be used to retrieve the pathame of the script. The documentation has been clarified regarding pre-defined macros such as ?MODULE and the module name.

Own Id: OTP-6593

Minor Makefile changes.

Own Id: OTP-6689 Aux Id: OTP-6742

STDLIB 1.14.4

Fixed Bugs and Malfunctions

The MD5 calculation of a BEAM file done by code:module_md5/1, beam_lib:md5/1, and by the compiler for the default value of the vsn attribute have all been changed so that its result will be the same on all platforms; modules containing funs could get different MD5s on different platforms.

Own Id: OTP-6459

When sorting terms using the file_sorter module (the option Format set to term), file errors were not always properly handled. This bug has been fixed.

The directory supplied with the tmpdir option is no longer checked unless it is actually used. The error reason not_a_directory can no longer be returned; instead a file_error tuple is returned

Own Id: OTP-6526

Bugs regarding try/catch have been fixed in the erl_eval module.

Own Id: OTP-6539

When sorting the operands of a join operation, QLC called file:open/3 with bad arguments. This bug has been fixed.

Own Id: OTP-6562 Aux Id: seq10606

Improvements and New Features

The functions beam_lib:cmp/1 and beam_lib:strip/1 (and similar functions) have been updated to handle optional chunks (such as "FunT") in more general way in order to be future compatible.

The function beam_lib:chunks/3 has been added.

The function beam_lib:md5/1 has been added.

Own Id: OTP-6443

Added base64 as a module to stdlib, encoding and decoding

Own Id: OTP-6470

Added the functions to_upper/1 and to_lower/1 to the string module. These provide case conversion for ISO/IEC 8859-1 characters (Latin1) and strings.

Own Id: OTP-6472

The callback function non_local_allowed/3 used by the restricted shell can now return the value {{restricted,NewFuncSpec,NewArgList},NewState} which can be used for letting the shell call some other function than the one specified.

Own Id: OTP-6497 Aux Id: seq10555

There is a new escript program that can be used for writing scripts in Erlang. Erlang scripts don't need to be compiled and any arguments can be passed to them without risk that they are interpreted by the Erlang system.

Own Id: OTP-6505

The Format argument of the functions io:fwrite/2,3 and io_lib:fwrite/2 is now allowed to be a binary.

Own Id: OTP-6517

STDLIB 1.14.3.1

Fixed Bugs and Malfunctions

The control sequences p and P of the Format argument of the functions io:fwrite/2,3 and io_lib:fwrite/2 could cause a badarg failure when applied to binaries. This bug was introduced in STDLIB 1.14.3. (Thanks to Denis Bilenko.)

Own Id: OTP-6495

Improvements and New Features

Added the option {cwd, Dir} to make zip-archives with relative pathnames without having to do (a global) file:set_cwd.

Own Id: OTP-6491 Aux Id: seq10551

STDLIB 1.14.3

Fixed Bugs and Malfunctions

The spawn_opt/2,3,4,5 option monitor -- introduced in Kernel 2.11.2 -- is currently not possible to use when starting a process using proc_lib, that is, also when starting a gen_server, gen_fsm etc.

This limitation has now been properly documented and the behavior of the gen_fsm, gen_server, and proc_lib start and start_link functions when providing this option has been changed from hanging indefinitely to failing with reason badarg.

(Thanks to Fredrik Linder)

Own Id: OTP-6345

Improvements and New Features

The control sequence P of the Format argument of the functions io:fwrite/2,3 and io_lib:fwrite/2 now replaces the tail of binary strings with ... when the maximum depth has been reached. For instance, io:fwrite("~P", [<<"a binary string">>, 3]). prints <<"a binary"...>>.

The indentation takes more care not to exceed the right margin, if possible.

If the maximum depth is reached while printing a tuple, ,... is printed instead of |... (this change applies to the control sequence W as well).

Own Id: OTP-6354

The Erlang shell command h/0 that prints the history list now avoids printing (huge) terms referred to by v/1 but instead just prints the call to v/1.

Own Id: OTP-6390

STDLIB 1.14.2.2

Fixed Bugs and Malfunctions

The functions dets:select/1,3, dets:match/1,3, and dets:match_object/1,3 have been changed as to never return {[],Continuation}. This change affects the corresponding functions in Mnesia.

Bugs have been fixed in QLC: qlc:info() could crash if the tmpdir option did not designate a valid directory; the results of looking up keys are kept in RAM, which should improve performance.

Own Id: OTP-6359

STDLIB 1.14.2.1

Fixed Bugs and Malfunctions

A bug in erl_pp:exprs() has been fixed.

Own Id: OTP-6321 Aux Id: seq10497

STDLIB 1.14.2

Fixed Bugs and Malfunctions

The control sequences p and P of the Format argument of the functions io:format/2,3 and io_lib:format/2 did not handle binaries very well. This bug, introduced in stdlib-1.14, has been fixed.

Own Id: OTP-6230

filelib:wildcard(Wc, PathWithRedundantSlashes), where PathWithRedundantSlashes is a directory path containing redundant slashes, such as /tmp/ or //tmp, could return incorrect results. (Thanks to Martin Bjorklund.)

Own Id: OTP-6271

The Erlang code preprocessor crashed if the predefined macros ?MODULE or ?MODULE_STRING were used before the module declaration. This bug has been fixed.

Own Id: OTP-6277

Improvements and New Features

Support for faster join of two tables has been added to the qlc module. There are two kinds of fast joins: lookup join that uses existing indices, and merge join that takes two sorted inputs. There is a new join option that can be used to force QLC to use a particular kind of join in some QLC expression.

Several other changes have also been included:

The new tmpdir option of cursor/2, eval/2, fold/4, and info/2 can be used to set the directory that join uses for temporary files. The option also overrides the tmpdir option of keysort/3 and sort/2.

The new lookup option can be used to assert that constants are looked up when evaluating some QLC expression.

The cache and cache_all options accept new tags: ets, list, and no. The tag list caches answers in a list using a temporary file if the answers cannot be held in RAM. Combining {cache,list} and {unique, true} is equivalent to calling sort/2 with the option unique set to true. The old tags true (equivalent to ets) and false (equivalent to no) are recognized for backward compatibility.

The new option max_list_size can be used to set the limit where merge join starts to use temporary files for large equivalence classes and when answers cached in lists are put on temporary files.

There is a new callback is_sorted_key to be supplied as an option to table/2.

QLC analyzes each and every QLC expression when trying to find constants for the lookup function. Hitherto only QLC expressions with exactly one generator were analyzed.

Note that only filters with guard syntax placed immediately after the generator are analyzed. The restriction to guard filters is an incompatible change. See qlc for further details.

In a similar way several match specifications for traversal of QLC tables can be utilized for different generators of one single QLC expression.

A bug has been fixed: when caching answers to a sufficiently complex query it could happen that some answers were not returned.

* POTENTIAL INCOMPATIBILITY *

Own Id: OTP-6038

The Erlang pretty printer (erl_pp) is now much faster when the code is deeply nested. A few minor bugs have been fixed as well.

Own Id: OTP-6227 Aux Id: OTP-5924

The Erlang shell now tries to garbage collect large binaries. Under certain circumstances such binaries could otherwise linger on for an indefinite amount of time.

Own Id: OTP-6239

To help Dialyzer find more bugs, many functions in the Kernel and STDLIB applications now only accept arguments of the type that is documented.

For instance, the functions lists:prefix/2 and lists:suffix/2 are documented to only accept lists as their arguments, but they actually accepted anything and returned false. That has been changed so that the functions cause an exception if one or both arguments are not lists.

Also, the string:strip/3 function is documented to take a character argument that is a character to strip from one or both ends of the string. Given a list instead of a character, it used to do nothing, but will now cause an exception.

Dialyzer will find most cases where those functions are passed arguments of the wrong type.

* POTENTIAL INCOMPATIBILITY *

Own Id: OTP-6295

STDLIB 1.14.1

Fixed Bugs and Malfunctions

The functions c:y/1,2 which call yecc:file/1,2 are now listed by c:help/0.

Documentation of c:y/1,2 has been added to c.

The fact that the control sequence character s recognizes binaries and deep character lists has been documented in io. This feature was added in R11B-0 (OTP-5403).

Own Id: OTP-6140

The shell command rr() sometimes failed to read record definitions from file(s). This problem has been fixed.

Own Id: OTP-6166 Aux Id: OTP-5878

The nonlocal function handler in erl_eval, which is used for implementing the restricted mode of the Erlang shell, did not handle calls to erlang:apply/3 correctly. This bug has been fixed.

Own Id: OTP-6169 Aux Id: seq10374

ets:rename/1 could deadlock, or crash the SMP emulator when the table wasn't a named table.

ets:next/2, and ets:prev/2 could return erroneous results on the SMP emulator.

Own Id: OTP-6198 Aux Id: seq10392, seq10415

When closing a Dets table the space management data was sometimes saved in such a way that opening the table could not be done without repairing the file. This bug has been fixed.

Own Id: OTP-6206

STDLIB 1.14

Fixed Bugs and Malfunctions

A bugfix in QLC: two of the call-back functions used for implementing QLC tables, TraverseFun and LookupFun, are now allowed to return a term other than a list. Such a term is immediately returned as the results of the current query, and is useful mostly for returning error tuples.

Several other minor bugs have been also been fixed.

Own Id: OTP-5195

The STDLIB modules error_logger_file_h and error_logger_tty_h now read the environment variable utc_log from the SASL application.

Own Id: OTP-5535

ets:info/1 has been corrected to behave according to the documentation and return a list of tuples, not a tuple with tuples.

* POTENTIAL INCOMPATIBILITY *

Own Id: OTP-5639

Referencing a so far undeclared record from the default value of some record declaration is from now on considered an error by the linter. It is also an error if the default value of a record declaration uses or binds a variable.

* POTENTIAL INCOMPATIBILITY *

Own Id: OTP-5878

When a file .hrl file is included using -include_lib, the include path is temporarily updated to include the directory the .hrl file was found in, which will allow that .hrl file to itself include files from the same directory as itself using -include. (Thanks to Richard Carlsson.)

Own Id: OTP-5944

Corrected filelib:ensure_dir/1 which sometimes returned true and sometimes ok to always return ok when successful. This goes against the documentation which said true, but ok was judged to be a more logical return value.

* POTENTIAL INCOMPATIBILITY *

Own Id: OTP-5960 Aux Id: seq10240

The shell now handles records better when used in calls on the form {Module, Function}(ArgList).

Own Id: OTP-5990 Aux Id: OTP-5876

The functions lists:ukeysort/2 and lists:ukeymerge/3 have been changed in such a way that two tuples are considered equal if their keys match.

For the sake of consistency, lists:usort/2 and lists:umerge/3 have been modified too: two elements are considered equal if they compare equal.

The file_sorter module has been modified in a similar way: the unique option now applies to the key (keysort() and keymerge()) and the ordering function (the option {order, Order}).

* POTENTIAL INCOMPATIBILITY *

Own Id: OTP-6019

Correction in documentation for ets:update_counter/3; failure with badarg also if the counter to be updated is the key.

Own Id: OTP-6072

When sorting terms using the file_sorter module and an ordering fun, the sort was not always stable. This bug has been fixed.

Own Id: OTP-6088

Improvements and New Features

Improvements of the linter:

The compile attribute is recognized after function definitions.

The new compiler option nowarn_deprecated_function can be used for turning off warnings for calls to deprecated functions.

The new compiler option {nowarn_unused_function,[{Name,Arity}]} turns off warnings for unused local functions for the mentioned functions. The new options {nowarn_deprecated_function,[{Module,Name,Arity}]} and {nowarn_bif_clash,[{Name,Arity}]} work similarly.

The Erlang code preprocessor epp now recognizes the file attribute. This attribute is meant to be used by tools such as Yecc that generate source code files.

Own Id: OTP-5362

The formatting option ~s of io:fwrite and io_lib:fwrite has been extended to handle arguments that are binaries or I/O lists.

Own Id: OTP-5403

The control sequences p and P of the Format argument of the functions io:format/2,3 and io_lib:format/2 have been changed as to display the contents of binaries containing printable characters as strings.

Own Id: OTP-5485

The linter emits warnings for functions exported more than once in export attributes.

Own Id: OTP-5494

A manual for STDLIB has been added, stdlib(6). It mentions the configuration parameters for the Erlang shell.

Own Id: OTP-5530

Added the zip module with functions for reading and creating zip archives. See zip.

Own Id: OTP-5786

Simple-one-for-one supervisors now store the pids of child processes using dict instead of a list. This significantly improves performance when there are many dynamic supervised child processes. (Thanks to Mickaël Rémond et al.)

Own Id: OTP-5898

When given the new option 'strict_record_tests', the compiler will generate code that verifies the record type for 'R#record.field' operations in guards. Code that verifies record types in bodies has already been generated since R10B, but in this release there will be a '{badrecord,RecordTag}' instead of a 'badmatch' if the record verification test fails. See the documentation for the compile module for more information.

The Erlang shell always applies strict record tests.

Own Id: OTP-5915 Aux Id: OTP-5714

The Erlang pretty printer (erl_pp) now tries to insert line breaks at appropriate places.

Own Id: OTP-5924

The public option has been removed from digraph:new/1. The reason is that several functions in the digraph module are implemented using multiple ETS accesses, which is not thread safe. (Thanks to Ulf Wiger.)

* POTENTIAL INCOMPATIBILITY *

Own Id: OTP-5985

The function lists:keyreplace/4 checks that the fourth argument (NewTuple) is a tuple.

Own Id: OTP-6023

Added an example of how to reconstruct source code from debug info (abstract code) to beam_lib. (Thanks to Mats Cronqvist who wrote the example.)

Own Id: OTP-6073

The new compiler option warn_unused_record is used for finding unused locally defined record types.

Own Id: OTP-6105

STDLIB 1.13.12

Fixed Bugs and Malfunctions

shell_default:xm/1 has been added. It calls xref:m/1.

Own Id: OTP-5405 Aux Id: OTP-4101

Warnings are output whenever so far undeclared records are referenced from some default value of a record declaration. In STDLIB 1.14 (R11B) such forward references will cause a compilation error.

Own Id: OTP-5878

The linter's check of the deprecated attribute did not take the compile option export_all into account. This bug has been fixed.

Own Id: OTP-5917

The Erlang pretty printer did not handle try/catch correctly. This bug has been fixed.

Own Id: OTP-5926

Corrected documentation for lists:nthtail/3.

Added documentation for lists:keymap/3.

Tried to clarify some other type declarations and function descriptions in lists.

Corrected documentation for timer:now_diff/2.

Fixed broken links in gen_fsm, gen_server, io_lib and lib(3).

Own Id: OTP-5931

Type checks have been added to functions in lists.erl.

Own Id: OTP-5939

Improvements and New Features

The new STDLIB module erl_expand_records expands records in abstract code. It is used by the Erlang shell, which means that Compiler is no longer used by the shell.

Own Id: OTP-5876 Aux Id: OTP-5435

The compiler will now warn that the megaco:format_versions/1 function is deprecated.

Own Id: OTP-5976

STDLIB 1.13.11

Fixed Bugs and Malfunctions

When calling gen_server:enter_loop with a registered server name, it was only checked that the registered name existed, not that it actually was the name of the calling process.

Own Id: OTP-5854

Improvements and New Features

More detail on beam_lib:version/1 in documentation.

Own Id: OTP-5789

The new function io:read/3 works like io:read/1,2 but takes a third argument, StartLine.

Own Id: OTP-5813

The new function gen_fsm:enter_loop/4,5,6, similar to gen_server:enter_loop/3,4,5, has been added.

Own Id: OTP-5846 Aux Id: seq10163

The function c:i/1 is now exported.

Own Id: OTP-5848 Aux Id: seq10164

STDLIB 1.13.10

Fixed Bugs and Malfunctions

A couple of type errors have been fixed in sofs.

Own Id: OTP-5739

The pre-processor used to complain that the macro definition '-define(S(S), ??S).' was circular, which it isn't. (Thanks to Richard Carlsson.)

Own Id: OTP-5777

STDLIB 1.13.9

Fixed Bugs and Malfunctions

The linter, QLC and the module erl_pp did not handle the new 'fun M:F/A' construct in all situations. This problem has been fixed.

Own Id: OTP-5644

Improvements and New Features

The manual pages for most of the Kernel and some of the STDLIB modules have been updated, in particular regarding type definitions.

The documentation of the return value for erts:info/1 has been corrected.

The documentation for erlang:statistics/1 now lists all possible arguments.

Own Id: OTP-5360

Replaced some tuple funs with the new fun M:F/A construct.

The high-order functions in the lists module no longer accept bad funs under any circumstances. 'lists:map(bad_fun, [])' used to return '[]' but now causes an exception.

Unused, broken compatibility code in the ets module was removed. (Thanks to Dialyzer.)

Eliminated 5 discrepancies found by Dialyzer in the Appmon application.

Own Id: OTP-5633

The c:i/0 function will now run in a paged mode if there are more than 100 processes in the system. (Thanks to Ulf Wiger.)

erlang:system_info(process_count) has been optimized and does now return exactly the same value as length(processes()). Previously erlang:system_info(process_count) did not include exiting processes which are included in length(processes()).

The +P flag for erl, which sets the maximum number of processes allowed to exist at the same, no longer accepts values higher than 134217727. (You will still probably run out of memory before you'll be able to reach that limit.)

Own Id: OTP-5645 Aux Id: seq9984

STDLIB 1.13.8

Fixed Bugs and Malfunctions

Very minor corrections in beam_lib and its documentation.

Own Id: OTP-5589

Improvements and New Features

The erlang:port_info/1 BIF is now documented. Minor corrections of the documentation for erlang:port_info/2.

Added a note to the documentation of the math module that all functions are not available on all platforms.

Added more information about the '+c' option in the erl man page in the ERTS documentation.

Own Id: OTP-5555

The new fun M:F/A construct creates a fun that refers to the latest version of M:F/A. This syntax is meant to replace tuple funs {M,F} which have many problems.

The new type test is_function(Fun,A) (which may be used in guards) test whether Fun is a fun that can be applied with A arguments. (Currently, Fun can also be a tuple fun.)

Own Id: OTP-5584

STDLIB 1.13.7

Fixed Bugs and Malfunctions

filelib:wildcard/2 was broken (it ignored its second argument).

Also, filelib:wildcard("Filename") (where the argument does not contain any meta-characters) would always return ["Filename"]. Corrected so that an empty list will be returned if "Filename" does not actually exist. (Same correction in filelib:wildcard/2.) (This change is a slight incompatibility.)

filelib:wildcard/1,2 will generate a different exception when given bad patterns such as "{a,". The exception used to be caused by 'exit(missing_delimiter)' but is now 'erlang:error({badpattern,missing_delimiter})'.

Own Id: OTP-5523 Aux Id: seq9824

Improvements and New Features

Further improvements of encrypted debug info: New option encrypt_debug_info for compiler.

Own Id: OTP-5541 Aux Id: seq9837

STDLIB 1.13.6

Fixed Bugs and Malfunctions

When opening a Dets table read only an attempt was sometimes made to re-hash the table resulting in an error message. This problem has been fixed.

Own Id: OTP-5487 Aux Id: OTP-4989

Improvements and New Features

It is now possible to encrypt the debug information in Beam files, to help keep the source code secret. See the documentation for compile on how to provide the key for encrypting, and the documentation for beam_lib on how to provide the key for decryption so that tools such as the Debugger, Xref, or Cover can be used.

The beam_lib:chunks/2 functions now accepts an additional chunk type compile_info to retrieve the compilation information directly as a term. (Thanks to Tobias Lindahl.)

Own Id: OTP-5460 Aux Id: seq9787

STDLIB 1.13.5

Fixed Bugs and Malfunctions

Closing a Dets table kept in RAM would cause a crash if the file could not be written. This problem has been fixed by returning an error tuple.

Own Id: OTP-5402

erl_pp now correctly pretty-prints fun F/A.

Own Id: OTP-5412

The Erlang shell failed if the compiler was not in the code path. This problem has been fixed, but in order to evaluate records the compiler is still needed.

Own Id: OTP-5435

Corrected the example in the documentation for ets:match/2. Also clarified that ets:update_counter/3 updates the counter atomically. (Thanks to Anders Svensson.)

Own Id: OTP-5452 Aux Id: seq9770, seq9789

Improvements and New Features

The possibility to start the Erlang shell in parallel with the rest of the system was reintroduced for backwards compatibility in STDLIB 1.13.1. The flag to be used for this is now called async_shell_start and has been documented. New shells started from the JCL menu are not synchronized with init anymore. This makes it possible to start a new shell (e.g. for debugging purposes) even if the initial shell has not come up.

Own Id: OTP-5406 Aux Id: OTP-5218

The compiler will now produce warnings when using the deprecated functions in the snmp module.

Own Id: OTP-5425

The function c:zi/0 has been removed. Use c:i/0 instead.

Own Id: OTP-5432

Corrected two minor bugs found by the Dialyzer: Calling a parameterized module from a restricted shell (i.e. if shell:start_restricted/1 has been used) would crash the shell evaluator. A debug printout in gen_fsm had a clause that would never match; causing less information to be printed.

And a somewhat more serious one also found by Dialyzer: rpc:yield/1 would crash unless the call started by rpc:async_call/4 had already finished; rpc:nb_yield(Key,infinity) would also crash.

Cleaned up and removed redundant code found by Dialyzer in erlang:dmonitor_p/2.

Own Id: OTP-5462

STDLIB 1.13.4

Fixed Bugs and Malfunctions

Bugs in the Erlang shell have been fixed.

Own Id: OTP-5327

Some dead code reported by Dialyzer was eliminated.

A bug in dbg when tracing to wrap trace files has been corrected. It failed to delete any already existing wrap trace files with the same names when starting a new wrap trace.

Own Id: OTP-5329

The linter could output invalid warnings about bit patterns in record initializations. This problem has been fixed.

Own Id: OTP-5338

ordsets:is_set(NoList), where NoList is any term except a list, would crash. For consistency with sets:is_set/1 and gb_sets:is_set/1, it now returns false.

Own Id: OTP-5341

A BIF erlang:raise/3 has been added. See the manual for details. It is intended for internal system programming only, advanced error handling.

Own Id: OTP-5376 Aux Id: OTP-5257

Improvements and New Features

The deprecated attribute is now checked by the linter. See xref for a description of the deprecated attribute.

Own Id: OTP-5276

The restricted shell will now indicate if the return value from a user predicate is on an incorrect form.

Own Id: OTP-5335

STDLIB 1.13.3

Fixed Bugs and Malfunctions

Bugs concerning unused and shadowed variables have been fixed in the linter.

Own Id: OTP-5091

A bug in the evaluator that caused the shell to choke on bit syntax expressions has been fixed.

Own Id: OTP-5237

io:format/2 et.al no longer crashes for some combinations of precision and value for format character "g". Previously it crashed if the precision P was 4 or lower and the absolute value of the float to print was lower than 10^4 but 10^(P-1) or higher. Now it will not crash depending on the value of the float.

Own Id: OTP-5263

Bugs in the handling of the bit syntax have been fixed in the Erlang shell.

Own Id: OTP-5269

gb_sets:del_element/2 was changed to do the same as gb_sets:delete_any/2 which was the original intention, not as gb_sets:delete/2. Code that relies on gb_sets:del_element/2 causing an error if the element does not exist must be changed to call gb_sets:delete/2 instead.

The documentation was also updated to explicitly document functions that were only referred to as 'aliases' of a documented function. Also, a list of all functions common to the gb_sets, sets, and ordsets was added.

* POTENTIAL INCOMPATIBILITY *

Own Id: OTP-5277

Debug messages have been removed from the QLC module.

Own Id: OTP-5283

Improvements and New Features

The size of continuations returned from dets:match/1,3, dets:match_object/1,3, and dets:select/1,3 has been reduced. This affects the amount of data Mnesia sends between nodes while evaluating QLC queries.

Own Id: OTP-5232

STDLIB 1.13.2

Improvements and New Features

The -rsh switch for starting a remote shell (introduced with OTP-5210) clashed with an already existing switch used by slave. Therefore the switch for the remote shell is now instead named -remsh.

Own Id: OTP-5248 Aux Id: OTP-5210

STDLIB 1.13.1

Fixed Bugs and Malfunctions

The Pman 'trace shell' functionality was broken as has now been fixed. Furthermore, Pman could not correctly find the pid of the active shell if more than one shell process was running on the node. This has also been corrected.

Own Id: OTP-5191

When the undocumented feature "parameterized modules" was used, the ?MODULE macro did not work correctly.

Own Id: OTP-5224

Improvements and New Features

You can now start Erlang with the -rsh flag which gives you a remote initial shell instead of a local one. Example:

            erl -sname this_node -rsh other_node@other_host

Own Id: OTP-5210

The man page for the lists module has been updated with description of the new zip, unzip, and partition/2 functions.

Own Id: OTP-5213

The top level group leader used to be listed as job #1 in the job list in JCL mode. Since there is no shell associated with this process that can be connected to, it will no longer be listed.

Own Id: OTP-5214

The possibility to start the Erlang shell in parallel with the rest of the system has been reintroduced for backwards compatibility. Note that this old behaviour is error prone and should not be used unless for some reason necessary.

Own Id: OTP-5218 Aux Id: seq9534

The shell commands rr/1,2,3 now accepts wildcards when reading record definitions from BEAM files.

Own Id: OTP-5226

Introduction

Scope

The Standard Erlang Libraries application, STDLIB, is mandatory in the sense that the minimal system based on Erlang/OTP consists of STDLIB and Kernel.

STDLIB contains the following functional areas:

Erlang shell

Command interface

Query interface

Interface to standard Erlang I/O servers

Interface to the Erlang built-in term storage BIFs

Regular expression matching functions for strings and binaries

Finite state machine

Event handling

Functions for the server of a client-server relation

Function to control applications in a distributed manner

Start and control of slave nodes

Operations on finite sets and relations represented as sets

Library for handling binary data

Disk-based term storage

List processing

Maps processing

Prerequisites

It is assumed that the reader is familiar with the Erlang programming language.

The Erlang I/O Protocol

The I/O protocol in Erlang enables bi-directional communication between clients and servers.

The I/O server is a process that handles the requests and performs the requested task on, for example, an I/O device.

The client is any Erlang process wishing to read or write data from/to the I/O device.

The common I/O protocol has been present in OTP since the beginning, but has been undocumented and has also evolved over the years. In an addendum to Robert Virding's rationale, the original I/O protocol is described. This section describes the current I/O protocol.

The original I/O protocol was simple and flexible. Demands for memory efficiency and execution time efficiency have triggered extensions to the protocol over the years, making the protocol larger and somewhat less easy to implement than the original. It can certainly be argued that the current protocol is too complex, but this section describes how it looks today, not how it should have looked.

The basic ideas from the original protocol still hold. The I/O server and client communicate with one single, rather simplistic protocol and no server state is ever present in the client. Any I/O server can be used together with any client code, and the client code does not need to be aware of the I/O device that the I/O server communicates with.

Protocol Basics

As described in Robert's paper, I/O servers and clients communicate using io_request/io_reply tuples as follows:

{io_request, From, ReplyAs, Request}
{io_reply, ReplyAs, Reply}

The client sends an io_request tuple to the I/O server and the server eventually sends a corresponding io_reply tuple.

From is the pid/0 of the client, the process which the I/O server sends the I/O reply to.

ReplyAs can be any datum and is returned in the corresponding io_reply. The io module monitors the I/O server and uses the monitor reference as the ReplyAs datum. A more complicated client can have many outstanding I/O requests to the same I/O server and can use different references (or something else) to differentiate among the incoming I/O replies. Element ReplyAs is to be considered opaque by the I/O server.

Notice that the pid/0 of the I/O server is not explicitly present in tuple io_reply. The reply can be sent from any process, not necessarily the actual I/O server.

Request and Reply are described below.

When an I/O server receives an io_request tuple, it acts upon the Request part and eventually sends an io_reply tuple with the corresponding Reply part.

Output Requests

To output characters on an I/O device, the following Requests exist:

{put_chars, Encoding, Characters}
{put_chars, Encoding, Module, Function, Args}

Encoding is unicode or latin1, meaning that the characters are (in case of binaries) encoded as UTF-8 or ISO Latin-1 (pure bytes). A well-behaved I/O server is also to return an error indication if list elements contain integers > 255 when Encoding is set to latin1.

Notice that this does not in any way tell how characters are to be put on the I/O device or handled by the I/O server. Different I/O servers can handle the characters however they want, this only tells the I/O server which format the data is expected to have. In the Module/Function/Args case, Encoding tells which format the designated function produces.

Notice also that byte-oriented data is simplest sent using the ISO Latin-1 encoding.

Characters are the data to be put on the I/O device. If Encoding is latin1, this is an iolist/0. If Encoding is unicode, this is an Erlang standard mixed Unicode list (one integer in a list per character, characters in binaries represented as UTF-8).

Module, Function, and Args denote a function that is called to produce the data (like io_lib:format/2).

Args is a list of arguments to the function. The function is to produce data in the specified Encoding. The I/O server is to call the function as apply(Mod, Func, Args) and put the returned data on the I/O device as if it was sent in a {put_chars, Encoding, Characters} request. If the function returns anything else than a binary or list, or throws an exception, an error is to be sent back to the client.

The I/O server replies to the client with an io_reply tuple, where element Reply is one of:

ok
{error, Error}

Error describes the error to the client, which can do whatever it wants with it. The io module typically returns it "as is".

Input Requests

To read characters from an I/O device, the following Requests exist:

{get_until, Encoding, Prompt, Module, Function, ExtraArgs}

Encoding denotes how data is to be sent back to the client and what data is sent to the function denoted by Module/Function/ExtraArgs. If the function supplied returns data as a list, the data is converted to this encoding. If the function supplied returns data in some other format, no conversion can be done, and it is up to the client-supplied function to return data in a proper way.

If Encoding is latin1, lists of integers 0..255 or binaries containing plain bytes are sent back to the client when possible. If Encoding is unicode, lists with integers in the whole Unicode range or binaries encoded in UTF-8 are sent to the client. The user-supplied function always sees lists of integers, never binaries, but the list can contain numbers > 255 if Encoding is unicode.

Prompt is a list of characters (not mixed, no binaries) or an atom to be output as a prompt for input on the I/O device. Prompt is often ignored by the I/O server; if set to '', it is always to be ignored (and results in nothing being written to the I/O device).

Module, Function, and ExtraArgs denote a function and arguments to determine when enough data is written. The function is to take two more arguments, the last state, and a list of characters. The function is to return one of:

{done, Result, RestChars}
{more, Continuation}

Result can be any Erlang term, but if it is a list/0, the I/O server can convert it to a binary/0 of appropriate format before returning it to the client, if the I/O server is set in binary mode (see below).

The function is called with the data the I/O server finds on its I/O device, returning one of:

{done, Result, RestChars} when enough data is read. In this case Result is sent to the client and RestChars is kept in the I/O server as a buffer for later input.

{more, Continuation}, which indicates that more characters are needed to complete the request.

Continuation is sent as the state in later calls to the function when more characters are available. When no more characters are available, the function must return {done, eof, Rest}. The initial state is the empty list. The data when an end of file is reached on the IO device is the atom eof.

An emulation of the get_line request can be (inefficiently) implemented using the following functions:

-module(demo).
-export([until_newline/3, get_line/1]).

until_newline(_ThisFar,eof,_MyStopCharacter) ->
    {done,eof,[]};
until_newline(ThisFar,CharList,MyStopCharacter) ->
    case
        lists:splitwith(fun(X) -> X =/= MyStopCharacter end,  CharList)
    of
  {L,[]} ->
            {more,ThisFar++L};
  {L2,[MyStopCharacter|Rest]} ->
      {done,ThisFar++L2++[MyStopCharacter],Rest}
    end.

get_line(IoServer) ->
    IoServer ! {io_request,
                self(),
                IoServer,
                {get_until, unicode, '', ?MODULE, until_newline, [$\n]}},
    receive
        {io_reply, IoServer, Data} ->
      Data
    end.

Notice that the last element in the Request tuple ([$\n]) is appended to the argument list when the function is called. The function is to be called like apply(Module, Function, [ State, Data | ExtraArgs ]) by the I/O server.

A fixed number of characters is requested using the following Request:

{get_chars, Encoding, Prompt, N}

Encoding and Prompt as for get_until.

N is the number of characters to be read from the I/O device.

A single line (as in former example) is requested with the following Request:

{get_line, Encoding, Prompt}

Encoding and Prompt as for get_until.

Clearly, get_chars and get_line could be implemented with the get_until request (and indeed they were originally), but demands for efficiency have made these additions necessary.

The I/O server replies to the client with an io_reply tuple, where element Reply is one of:

Data
eof
{error, Error}

Data is the characters read, in list or binary form (depending on the I/O server mode, see the next section).

eof is returned when input end is reached and no more data is available to the client process.

Error describes the error to the client, which can do whatever it wants with it. The io module typically returns it as is.

I/O Server Modes

Demands for efficiency when reading data from an I/O server has not only lead to the addition of the get_line and get_chars requests, but has also added the concept of I/O server options. No options are mandatory to implement, but all I/O servers in the Erlang standard libraries honor the binary option, which allows element Data of the io_reply tuple to be a binary instead of a list when possible. If the data is sent as a binary, Unicode data is sent in the standard Erlang Unicode format, that is, UTF-8 (notice that the function of the get_until request still gets list data regardless of the I/O server mode).

Notice that the get_until request allows for a function with the data specified as always being a list. Also, the return value data from such a function can be of any type (as is indeed the case when an io:fread/2,3 request is sent to an I/O server). The client must be prepared for data received as answers to those requests to be in various forms. However, the I/O server is to convert the results to binaries whenever possible (that is, when the function supplied to get_until returns a list). This is done in the example in section An Annotated and Working Example I/O Server.

An I/O server in binary mode affects the data sent to the client, so that it must be able to handle binary data. For convenience, the modes of an I/O server can be set and retrieved using the following I/O requests:

{setopts, Opts}

Opts is a list of options in the format recognized by the proplists module (and by the I/O server).

As an example, the I/O server for the interactive shell (in group.erl) understands the following options:

{binary, boolean()} (or binary/list)
{echo, boolean()}
{expand_fun, fun()}
{encoding, unicode/latin1} (or unicode/latin1)

Options binary and encoding are common for all I/O servers in OTP, while echo and expand are valid only for this I/O server. Option unicode notifies how characters are put on the physical I/O device, that is, if the terminal itself is Unicode-aware. It does not affect how characters are sent in the I/O protocol, where each request contains encoding information for the provided or returned data.

The I/O server is to send one of the following as Reply:

ok
{error, Error}

An error (preferably enotsup) is to be expected if the option is not supported by the I/O server (like if an echo option is sent in a setopts request to a plain file).

To retrieve options, the following request is used:

getopts

This request asks for a complete list of all options supported by the I/O server as well as their current values.

The I/O server replies:

OptList
{error, Error}

OptList is a list of tuples {Option, Value}, where Option always is an atom.

Multiple I/O Requests

The Request element can in itself contain many Requests by using the following format:

{requests, Requests}

Requests is a list of valid io_request tuples for the protocol. They must be executed in the order that they appear in the list. The execution is to continue until one of the requests results in an error or the list is consumed. The result of the last request is sent back to the client.

The I/O server can, for a list of requests, send any of the following valid results in the reply, depending on the requests in the list:

ok
{ok, Data}
{ok, Options}
{error, Error}

Optional I/O Request

The following I/O request is optional to implement and a client is to be prepared for an error return:

{get_geometry, Geometry}

Geometry is the atom rows or the atom columns.

The I/O server is to send one of the following as Reply:

N
{error, Error}

N is the number of character rows or columns that the I/O device has, if applicable to the I/O device handled by the I/O server, otherwise {error, enotsup} is a good answer.

Unimplemented Request Types

If an I/O server encounters a request that it does not recognize (that is, the io_request tuple has the expected format, but the Request is unknown), the I/O server is to send a valid reply with the error tuple:

{error, request}

This makes it possible to extend the protocol with optional requests and for the clients to be somewhat backward compatible.

An Annotated and Working Example I/O Server

An I/O server is any process capable of handling the I/O protocol. There is no generic I/O server behavior, but could well be. The framework is simple, a process handling incoming requests, usually both I/O-requests and other I/O device-specific requests (positioning, closing, and so on).

The example I/O server stores characters in an ETS table, making up a fairly crude RAM file.

The module begins with the usual directives, a function to start the I/O server and a main loop handling the requests:

-module(ets_io_server).

-export([start_link/0, init/0, loop/1, until_newline/3, until_enough/3]).

-define(CHARS_PER_REC, 10).

-record(state, {
	  table,
	  position, % absolute
	  mode % binary | list
	 }).

start_link() ->
    spawn_link(?MODULE,init,[]).

init() ->
    Table = ets:new(noname,[ordered_set]),
    ?MODULE:loop(#state{table = Table, position = 0, mode=list}).

loop(State) ->
    receive
	{io_request, From, ReplyAs, Request} ->
	    case request(Request,State) of
		{Tag, Reply, NewState} when Tag =:= ok; Tag =:= error ->
		    reply(From, ReplyAs, Reply),
		    ?MODULE:loop(NewState);
		{stop, Reply, _NewState} ->
		    reply(From, ReplyAs, Reply),
		    exit(Reply)
	    end;
	%% Private message
	{From, rewind} ->
	    From ! {self(), ok},
	    ?MODULE:loop(State#state{position = 0});
	_Unknown ->
	    ?MODULE:loop(State)
    end.

The main loop receives messages from the client (which can use the the io module to send requests). For each request, the function request/2 is called and a reply is eventually sent using function reply/3.

The "private" message {From, rewind} results in the current position in the pseudo-file to be reset to 0 (the beginning of the "file"). This is a typical example of I/O device-specific messages not being part of the I/O protocol. It is usually a bad idea to embed such private messages in io_request tuples, as that can confuse the reader.

First, we examine the reply function:

reply(From, ReplyAs, Reply) ->
    From ! {io_reply, ReplyAs, Reply}.

It sends the io_reply tuple back to the client, providing element ReplyAs received in the request along with the result of the request, as described earlier.

We need to handle some requests. First the requests for writing characters:

request({put_chars, Encoding, Chars}, State) ->
    put_chars(unicode:characters_to_list(Chars,Encoding),State);
request({put_chars, Encoding, Module, Function, Args}, State) ->
    try
	request({put_chars, Encoding, apply(Module, Function, Args)}, State)
    catch
	_:_ ->
	    {error, {error,Function}, State}
    end;

The Encoding says how the characters in the request are represented. We want to store the characters as lists in the ETS table, so we convert them to lists using function unicode:characters_to_list/2. The conversion function conveniently accepts the encoding types unicode and latin1, so we can use Encoding directly.

When Module, Function, and Arguments are provided, we apply it and do the same with the result as if the data was provided directly.

We handle the requests for retrieving data:

request({get_until, Encoding, _Prompt, M, F, As}, State) ->
    get_until(Encoding, M, F, As, State);
request({get_chars, Encoding, _Prompt, N}, State) ->
    %% To simplify the code, get_chars is implemented using get_until
    get_until(Encoding, ?MODULE, until_enough, [N], State);
request({get_line, Encoding, _Prompt}, State) ->
    %% To simplify the code, get_line is implemented using get_until
    get_until(Encoding, ?MODULE, until_newline, [$\n], State);

Here we have cheated a little by more or less only implementing get_until and using internal helpers to implement get_chars and get_line. In production code, this can be inefficient, but that depends on the frequency of the different requests. Before we start implementing functions put_chars/2 and get_until/5, we examine the few remaining requests:

request({get_geometry,_}, State) ->
    {error, {error,enotsup}, State};
request({setopts, Opts}, State) ->
    setopts(Opts, State);
request(getopts, State) ->
    getopts(State);
request({requests, Reqs}, State) ->
     multi_request(Reqs, {ok, ok, State});

Request get_geometry has no meaning for this I/O server, so the reply is {error, enotsup}. The only option we handle is binary/list, which is done in separate functions.

The multi-request tag (requests) is handled in a separate loop function applying the requests in the list one after another, returning the last result.

{error, request} must be returned if the request is not recognized:

request(_Other, State) ->
    {error, {error, request}, State}.

Next we handle the different requests, first the fairly generic multi-request type:

multi_request([R|Rs], {ok, _Res, State}) ->
    multi_request(Rs, request(R, State));
multi_request([_|_], Error) ->
    Error;
multi_request([], Result) ->
    Result.

We loop through the requests one at the time, stopping when we either encounter an error or the list is exhausted. The last return value is sent back to the client (it is first returned to the main loop and then sent back by function io_reply).

Requests getopts and setopts are also simple to handle. We only change or read the state record:

setopts(Opts0,State) ->
    Opts = proplists:unfold(
	     proplists:substitute_negations(
	       [{list,binary}],
	       Opts0)),
    case check_valid_opts(Opts) of
	true ->
	        case proplists:get_value(binary, Opts) of
		    true ->
			{ok,ok,State#state{mode=binary}};
		    false ->
			{ok,ok,State#state{mode=binary}};
		    _ ->
			{ok,ok,State}
		end;
	false ->
	    {error,{error,enotsup},State}
    end.
check_valid_opts([]) ->
    true;
check_valid_opts([{binary,Bool}|T]) when is_boolean(Bool) ->
    check_valid_opts(T);
check_valid_opts(_) ->
    false.

getopts(#state{mode=M} = S) ->
    {ok,[{binary, case M of
		      binary ->
			  true;
		      _ ->
			  false
		  end}],S}.

As a convention, all I/O servers handle both {setopts, [binary]}, {setopts, [list]}, and {setopts,[{binary, boolean()}]}, hence the trick with proplists:substitute_negations/2 and proplists:unfold/1. If invalid options are sent to us, we send {error, enotsup} back to the client.

Request getopts is to return a list of {Option, Value} tuples. This has the twofold function of providing both the current values and the available options of this I/O server. We have only one option, and hence return that.

So far this I/O server is fairly generic (except for request rewind handled in the main loop and the creation of an ETS table). Most I/O servers contain code similar to this one.

To make the example runnable, we start implementing the reading and writing of the data to/from the ETS table. First function put_chars/3:

put_chars(Chars, #state{table = T, position = P} = State) ->
    R = P div ?CHARS_PER_REC,
    C = P rem ?CHARS_PER_REC,
    [ apply_update(T,U) || U <- split_data(Chars, R, C) ],
    {ok, ok, State#state{position = (P + length(Chars))}}.

We already have the data as (Unicode) lists and therefore only split the list in runs of a predefined size and put each run in the table at the current position (and forward). Functions split_data/3 and apply_update/2 are implemented below.

Now we want to read data from the table. Function get_until/5 reads data and applies the function until it says that it is done. The result is sent back to the client:

get_until(Encoding, Mod, Func, As,
	  #state{position = P, mode = M, table = T} = State) ->
    case get_loop(Mod,Func,As,T,P,[]) of
	{done,Data,_,NewP} when is_binary(Data); is_list(Data) ->
	    if
		M =:= binary ->
		    {ok,
		     unicode:characters_to_binary(Data, unicode, Encoding),
		     State#state{position = NewP}};
		true ->
		    case check(Encoding,
		               unicode:characters_to_list(Data, unicode))
                    of
			{error, _} = E ->
			    {error, E, State};
			List ->
			    {ok, List,
			     State#state{position = NewP}}
		    end
	    end;
	{done,Data,_,NewP} ->
	    {ok, Data, State#state{position = NewP}};
	Error ->
	    {error, Error, State}
    end.

get_loop(M,F,A,T,P,C) ->
    {NewP,L} = get(P,T),
    case catch apply(M,F,[C,L|A]) of
	{done, List, Rest} ->
	    {done, List, [], NewP - length(Rest)};
	{more, NewC} ->
	    get_loop(M,F,A,T,NewP,NewC);
	_ ->
	    {error,F}
    end.

Here we also handle the mode (binary or list) that can be set by request setopts. By default, all OTP I/O servers send data back to the client as lists, but switching mode to binary can increase efficiency if the I/O server handles it in an appropriate way. The implementation of get_until is difficult to get efficient, as the supplied function is defined to take lists as arguments, but get_chars and get_line can be optimized for binary mode. However, this example does not optimize anything.

It is important though that the returned data is of the correct type depending on the options set. We therefore convert the lists to binaries in the correct encoding if possible before returning. The function supplied in the get_until request tuple can, as its final result return anything, so only functions returning lists can get them converted to binaries. If the request contains encoding tag unicode, the lists can contain all Unicode code points and the binaries are to be in UTF-8. If the encoding tag is latin1, the client is only to get characters in the range 0..255. Function check/2 takes care of not returning arbitrary Unicode code points in lists if the encoding was specified as latin1. If the function does not return a list, the check cannot be performed and the result is that of the supplied function untouched.

To manipulate the table we implement the following utility functions:

check(unicode, List) ->
    List;
check(latin1, List) ->
    try
	[ throw(not_unicode) || X <- List,
				X > 255 ],
	List
    catch
	throw:_ ->
	    {error,{cannot_convert, unicode, latin1}}
    end.

The function check provides an error tuple if Unicode code points > 255 are to be returned if the client requested latin1.

The two functions until_newline/3 and until_enough/3 are helpers used together with function get_until/5 to implement get_chars and get_line (inefficiently):

until_newline([],eof,_MyStopCharacter) ->
    {done,eof,[]};
until_newline(ThisFar,eof,_MyStopCharacter) ->
    {done,ThisFar,[]};
until_newline(ThisFar,CharList,MyStopCharacter) ->
    case
        lists:splitwith(fun(X) -> X =/= MyStopCharacter end,  CharList)
    of
	{L,[]} ->
            {more,ThisFar++L};
	{L2,[MyStopCharacter|Rest]} ->
	    {done,ThisFar++L2++[MyStopCharacter],Rest}
    end.

until_enough([],eof,_N) ->
    {done,eof,[]};
until_enough(ThisFar,eof,_N) ->
    {done,ThisFar,[]};
until_enough(ThisFar,CharList,N)
  when length(ThisFar) + length(CharList) >= N ->
    {Res,Rest} = my_split(N,ThisFar ++ CharList, []),
    {done,Res,Rest};
until_enough(ThisFar,CharList,_N) ->
    {more,ThisFar++CharList}.

As can be seen, the functions above are just the type of functions that are to be provided in get_until requests.

To complete the I/O server, we only need to read and write the table in an appropriate way:

get(P,Tab) ->
    R = P div ?CHARS_PER_REC,
    C = P rem ?CHARS_PER_REC,
    case ets:lookup(Tab,R) of
	[] ->
	    {P,eof};
	[{R,List}] ->
	    case my_split(C,List,[]) of
		{_,[]} ->
		    {P+length(List),eof};
		{_,Data} ->
		    {P+length(Data),Data}
	    end
    end.

my_split(0,Left,Acc) ->
    {lists:reverse(Acc),Left};
my_split(_,[],Acc) ->
    {lists:reverse(Acc),[]};
my_split(N,[H|T],Acc) ->
    my_split(N-1,T,[H|Acc]).

split_data([],_,_) ->
    [];
split_data(Chars, Row, Col) ->
    {This,Left} = my_split(?CHARS_PER_REC - Col, Chars, []),
    [ {Row, Col, This} | split_data(Left, Row + 1, 0) ].

apply_update(Table, {Row, Col, List}) ->
    case ets:lookup(Table,Row) of
	[] ->
	    ets:insert(Table,{Row, lists:duplicate(Col,0) ++ List});
	[{Row, OldData}] ->
	    {Part1,_} = my_split(Col,OldData,[]),
	    {_,Part2} = my_split(Col+length(List),OldData,[]),
	    ets:insert(Table,{Row, Part1 ++ List ++ Part2})
    end.

The table is read or written in chunks of ?CHARS_PER_REC, overwriting when necessary. The implementation is clearly not efficient, it is just working.

This concludes the example. It is fully runnable and you can read or write to the I/O server by using, for example, the io module or even the file module. It is as simple as that to implement a fully fledged I/O server in Erlang.

Creating a custom shell

This guide will show how to create a custom shell. The most common use case for this is to support other languages running on the Erlang VM, but it can also be used to create specialized debugging shells a system.

This guide will build on top of the built-in Erlang line editor, which means that the keybindings described in tty - A Command-Line Interface can be used edit the input before it is passed to the custom shell. This somewhat limits what the custom shell can do, but it also means that we do not have to implement line editing ourselves. If you need more control over the shell, then use Creating a terminal application as a starting-point to build your own line editor and shell.

A process inspection shell

The custom shell that we are going to build is a process inspection shell that supports the following commands:

list - lists all processes

inspect pid() - inspect a process

suspend pid() - suspend a process

resume pid() - resume a process

Lets get started!

Starting with a custom shell

The custom shell will be implemented in an escript, but it could just as well be in a regular system or as a remote shell. To start a custom shell we first need to start Erlang in -noinput or -noshell mode. escript are started by default in -noshell mode, so we don't have to do anything special here. To start the custom shell we then call shell:start_interactive/1.

#!/usr/bin/env escript
%% pshell.es
-export([start/0]).
main(_Args) ->
    shell:start_interactive({?MODULE, start, []}),
    timer:sleep(infinity). %% Make sure the escript does not exit

-spec start() -> pid().
start() ->
    spawn(fun() ->
                  io:format(~"Starting process inspection shell~n"),
                  loop()
          end).

loop() ->
    receive _M -> loop() end.

If we run the above we will get this:

$ ./pshell.es
Erlang/OTP 28 [DEVELOPMENT] [erts-15.0.1] [source-b395339a02] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] [jit:ns]

Starting process inspection shell

The io:standard_io/0 of the created shell process will be set to the Erlang line editor, which means that we can use the normal io functions to read and write data to the terminal.

Adding our first command

Let's start adding the shell interface. We will use io:get_line/1 to read from io:standard_io/0 as this shell will be line based. However, for a more complex shell it is better to send get_until I/O requests as commands read that way can span multiple lines. So we expand our loop/0 with a io:get_line/1 and pass the results to our parser.

loop() ->
    case io:get_line("> ") of
        eof -> ok;
        {error, Reason} -> exit(Reason);
        Data -> eval(string:trim(Data))
    end,
    loop().

eval("list") ->
    Format = " ~.10ts | ~.10ts | ~.10ts~n",
    io:format(Format,["Pid", "Name", "MsgQ Len"]),
    [begin
         [{registered_name,Name},{message_queue_len,Len}]
             = erlang:process_info(Pid, [registered_name, message_queue_len]),
         io:format(Format,[to_list(Pid), to_list(Name), to_list(Len)])
     end || Pid <- processes()];
eval(Unknown) ->
    io:format("Unknown command: '~ts'~n",[Unknown]).

to_list(Pid) when is_pid(Pid) ->
    pid_to_list(Pid);
to_list(Atom) when is_atom(Atom) ->
    atom_to_list(Atom);
to_list(Int) when is_integer(Int) ->
    integer_to_list(Int);
to_list(List) when is_list(List) ->
    List.

If we run the above we will get this:

$ ./pshell.es
Erlang/OTP 28 [DEVELOPMENT] [erts-15.0.1] [source-b395339a02] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] [jit:ns]

Starting process inspection shell
> list
 Pid        | Name       | MsgQ Len
 <0.0.0>    | init       | 0
 <0.1.0>    | erts_code_ | 0
 <0.2.0>    |            | 0
 <0.3.0>    |            | 0
 <0.4.0>    |            | 0
 <0.5.0>    |            | 0
 <0.6.0>    |            | 0
 <0.7.0>    |            | 0
 <0.8.0>    | socket_reg | 0
 <0.10.0>   |            | 0
 <0.11.0>   | erl_prim_l | 0
 <0.43.0>   | logger     | 0
 <0.45.0>   | applicatio | 0
...

With this all in place we can now easily add inspect, suspend and resume as well.

eval("inspect " ++ PidStr) ->
    case parse_pid(PidStr) of
        invalid -> ok;
        Pid ->
            [{registered_name, Name}, {memory, Memory}, {messages, Messages}, {status, Status}] =
                erlang:process_info(Pid, [registered_name, memory, messages, status]),
            io:format("Pid: ~p~nName: ~ts~nStatus: ~p~nMemory: ~p~nMessages: ~p~n",
                      [Pid, to_list(Name), Status, Memory, Messages])
    end;
eval("suspend " ++ PidStr) ->
    case parse_pid(PidStr) of
        invalid -> ok;
        Pid ->
            erlang:suspend_process(Pid),
            io:format("Suspeneded ~ts~n")
    end;
eval("resume " ++ PidStr) ->
    case parse_pid(PidStr) of
        invalid -> ok;
        Pid ->
            erlang:resumne_process(Pid),
            io:format("Resumed ~ts~n")
    end;

Adding autocompletion

Wouldn't it be great if we could add some simple auto-completion for our shell? We can do that by setting a edlin_expand fun for our shell. This is done by calling io:setopts([{expand_fun, Fun}]). The fun that we provide is will receive the reversed current line from edlin and is expected to return possible expansions. Let's start by adding a simple fun to expand our commands.

-spec start() -> pid().
start() ->
    spawn(fun() ->
                  io:setopts([{expand_fun, fun expand_fun/1}]),
                  io:format(~"Starting process inspection shell~n"),
                  loop()
          end).

-spec expand_fun(ReverseLine :: string()) -> {yes, string(), list(string())} |
          {no, nil(), nil()}.
expand_fun("") -> %% If line is empty, we list all available commands
    {yes, "", ["list", "inspect", "suspend", "resume"]};
expand_fun(Curr) ->
    expand_fun(lists:reverse(Curr), ["list", "inspect", "suspend", "resume"]).

expand_fun(_Curr, []) ->
    {no, "", []};
expand_fun(Curr, [Cmd | T]) ->
    case lists:prefix(Curr, Cmd) of
        true ->
            %% If Curr is a prefix of Cmd we subtract Curr from Cmd to get the
            %% characters we need to complete with.
            {yes, lists:reverse(lists:reverse(Cmd) -- lists:reverse(Curr)), []};
        false ->
            expand_fun(Curr, T)
    end.

With the above code we will get expansions of our commands if we hit <TAB> in the shell. Its possible to make very complex completion algorithms, for example the Erlang shell has completions based on the function specifications of your code. It is important though that the shell still feels responsive, so calling out to a LLM model for completion may or may not be a good idea.

The complete source code for this example can be found here.

Creating a terminal application

This guide will show how to create a very simple tic-tac-toe game in the shell. We will go through how to read key-strokes and how to update the screen to show the tic-tac-toe board. The game will be implemented as an escript, but it can just as well be implemented in a regular system.

Let us start by drawing the board which will look like this:

╔═══════╤═══════╤═══════╗
║┌─────┐│       │       ║
║│     ││       │       ║     Place an X by pressing Enter
║└─────┘│       │       ║
╟───────┼───────┼───────╢
║       │       │       ║
║       │       │       ║
║       │       │       ║
╟───────┼───────┼───────╢
║       │       │       ║
║       │       │       ║
║       │       │       ║
╚═══════╧═══════╧═══════╝

We will use the alternate screen buffer for our game so first we need to set that up:

#!/usr/bin/env escript
main(_Args) ->

    io:put_chars("\e[?1049h"), %% Enable alternate screen buffer
    io:put_chars("\e[?25l"), %% Hide the cursor
    draw_board(),
    timer:sleep(5000),
    io:put_chars("\e[?25h"), %% Show the cursor
    io:put_chars("\e[?1049l"), %% Disable alternate screen buffer
    ok.

We then use the box drawing parts of Unicode to draw our board:

draw_board() ->
    io:put_chars("\e[5;0H"), %% Move cursor to top left
    io:put_chars(
      ["     ╔═══════╤═══════╤═══════╗\r\n",
       "     ║       │       │       ║\r\n",
       "     ║       │       │       ║     Place an X by pressing Enter\r\n",
       "     ║       │       │       ║\r\n",
       "     ╟───────┼───────┼───────╢\r\n",
       "     ║       │       │       ║\r\n",
       "     ║       │       │       ║\r\n",
       "     ║       │       │       ║\r\n",
       "     ╟───────┼───────┼───────╢\r\n",
       "     ║       │       │       ║\r\n",
       "     ║       │       │       ║\r\n",
       "     ║       │       │       ║\r\n",
       "     ╚═══════╧═══════╧═══════╝\r\n"]),
    ok.

Let us add some interactivity to our game! To do that we need to change the shell from running in cooked to raw mode. This is done by calling shell:start_interactive({noshell, raw}). We can then use io:get_chars/2 to read key strokes from the user. The key strokes will be returned as ANSI escape codes, so we will have need to handle the codes for up, down, left, right and enter.

It could look something like this:

main(_Args) ->
    ok = shell:start_interactive({noshell, raw}),

    io:put_chars("\e[?1049h"), %% Enable alternate screen buffer
    io:put_chars("\e[?25l"), %% Hide the cursor
    draw_board(),
    loop(0),
    io:put_chars("\e[?25h"), %% Show the cursor
    io:put_chars("\e[?1049l"), %% Disable alternate screen buffer
    ok.

loop(Pos) ->
    io:put_chars(draw_selection(Pos)),
    %% Read at most 1024 characters from stdin.
    Chars = io:get_chars("", 1024),
    case handle_input(Chars, Pos) of
        stop -> stop;
        NewPos ->
            io:put_chars(clear_selection(Pos)),
            loop(NewPos)
    end.

handle_input("\e[A" ++ Rest, Pos) ->
    %% Up key
    handle_input(Rest, max(0, Pos - 3));
handle_input("\e[B" ++ Rest, Pos) ->
    %% Down key
    handle_input(Rest, min(8, Pos + 3));
handle_input("\e[C" ++ Rest, Pos) ->
    %% right key
    handle_input(Rest, min(8, Pos + 1));
handle_input("\e[D" ++ Rest, Pos) ->
    %% left key
    handle_input(Rest, max(0, Pos - 1));
handle_input("q" ++ _, _State) ->
    stop;
handle_input([_ | T], State) ->
    handle_input(T, State);
handle_input([], State) ->
    State.

Note that when using io:get_chars/2 with the shell set in {noshell, raw} mode it will return as soon as any data is available. The number of characters is the maximum number that will be returned. We use 1024 here to make sure that we always get all the data in one read.

We also need to draw the selection marker, we do this using some simple drawing routines.

%% Clear/draw the selection markers, making sure
%% not to overwrite if a X or O exists.
%%   \b = Move cursor left
%%   \e[C = Move cursor right
%%   \n = Move cursor down
clear_selection(Pos) ->
    [set_position(Pos),
     "       ","\b\b\b\b\b\b\b\n",
     " \e[C\e[C\e[C\e[C\e[C ",
     "\b\b\b\b\b\b\b\n","       "].

draw_selection(Pos) ->
    [set_position(Pos),
     "┌─────┐","\b\b\b\b\b\b\b\n",
     "│\e[C\e[C\e[C\e[C\e[C│",
     "\b\b\b\b\b\b\b\n","└─────┘"].

%% Set the cursor position to be at the top
%% left of the field of the given position
set_position(Pos) ->
    Row = 6 + (Pos div 3) * 4,
    Col = 7 + (Pos rem 3) * 8,
    io_lib:format("\e[~p;~pH",[Row, Col]).

Now we have a program where we can move the marker around the board. To complete the game we need to add some state so that we know which squares are marked and whos turn it is. You can find the final solution in tic-tac-toe.es.

Using Unicode in Erlang

Unicode Implementation

Implementing support for Unicode character sets is an ongoing process. The Erlang Enhancement Proposal (EEP) 10 outlined the basics of Unicode support and specified a default encoding in binaries that all Unicode-aware modules are to handle in the future.

Here is an overview what has been done so far:

The functionality described in EEP10 was implemented in Erlang/OTP R13A.

Erlang/OTP R14B01 added support for Unicode filenames, but it was not complete and was by default disabled on platforms where no guarantee was given for the filename encoding.

With Erlang/OTP R16A came support for UTF-8 encoded source code, with enhancements to many of the applications to support both Unicode encoded filenames and support for UTF-8 encoded files in many circumstances. Most notable is the support for UTF-8 in files read by file:consult/1, release handler support for UTF-8, and more support for Unicode character sets in the I/O system.

In Erlang/OTP 17.0, the encoding default for Erlang source files was switched to UTF-8.

In Erlang/OTP 20.0, atoms and function can contain Unicode characters. Module names, application names, and node names are still restricted to the ISO Latin-1 range.

Support was added for normalizations forms in unicode and the string module now handles utf8-encoded binaries.

This section outlines the current Unicode support and gives some recipes for working with Unicode data.

Understanding Unicode

Experience with the Unicode support in Erlang has made it clear that understanding Unicode characters and encodings is not as easy as one would expect. The complexity of the field and the implications of the standard require thorough understanding of concepts rarely before thought of.

Also, the Erlang implementation requires understanding of concepts that were never an issue for many (Erlang) programmers. To understand and use Unicode characters requires that you study the subject thoroughly, even if you are an experienced programmer.

As an example, contemplate the issue of converting between upper and lower case letters. Reading the standard makes you realize that there is not a simple one to one mapping in all scripts, for example:

In German, the letter "ß" (sharp s) is in lower case, but the uppercase equivalent is "SS".

In Greek, the letter "Σ" has two different lowercase forms, "ς" in word-final position and "σ" elsewhere.

In Turkish, both dotted and dotless "i" exist in lower case and upper case forms.

Cyrillic "I" has usually no lowercase form.

Languages with no concept of upper case (or lower case).

So, a conversion function must know not only one character at a time, but possibly the whole sentence, the natural language to translate to, the differences in input and output string length, and so on. Erlang/OTP has currently no Unicode uppercase/lowercase functionality with language specific handling, but publicly available libraries address these issues.

Another example is the accented characters, where the same glyph has two different representations. The Swedish letter "ö" is one example. The Unicode standard has a code point for it, but you can also write it as "o" followed by "U+0308" (Combining Diaeresis, with the simplified meaning that the last letter is to have "¨" above). They have the same glyph, user perceived character. They are for most purposes the same, but have different representations. For example, MacOS X converts all filenames to use Combining Diaeresis, while most other programs (including Erlang) try to hide that by doing the opposite when, for example, listing directories. However it is done, it is usually important to normalize such characters to avoid confusion.

The list of examples can be made long. One need a kind of knowledge that was not needed when programs only considered one or two languages. The complexity of human languages and scripts has certainly made this a challenge when constructing a universal standard. Supporting Unicode properly in your program will require effort.

What Unicode Is

Unicode is a standard defining code points (numbers) for all known, living or dead, scripts. In principle, every symbol used in any language has a Unicode code point. Unicode code points are defined and published by the Unicode Consortium, which is a non-profit organization.

Support for Unicode is increasing throughout the world of computing, as the benefits of one common character set are overwhelming when programs are used in a global environment. Along with the base of the standard, the code points for all the scripts, some encoding standards are available.

It is vital to understand the difference between encodings and Unicode characters. Unicode characters are code points according to the Unicode standard, while the encodings are ways to represent such code points. An encoding is only a standard for representation. UTF-8 can, for example, be used to represent a very limited part of the Unicode character set (for example ISO-Latin-1) or the full Unicode range. It is only an encoding format.

As long as all character sets were limited to 256 characters, each character could be stored in one single byte, so there was more or less only one practical encoding for the characters. Encoding each character in one byte was so common that the encoding was not even named. With the Unicode system there are much more than 256 characters, so a common way is needed to represent these. The common ways of representing the code points are the encodings. This means a whole new concept to the programmer, the concept of character representation, which was a non-issue earlier.

Different operating systems and tools support different encodings. For example, Linux and MacOS X have chosen the UTF-8 encoding, which is backward compatible with 7-bit ASCII and therefore affects programs written in plain English the least. Windows supports a limited version of UTF-16, namely all the code planes where the characters can be stored in one single 16-bit entity, which includes most living languages.

The following are the most widely spread encodings:

Bytewise representation - This is not a proper Unicode representation, but the representation used for characters before the Unicode standard. It can still be used to represent character code points in the Unicode standard with numbers < 256, which exactly corresponds to the ISO Latin-1 character set. In Erlang, this is commonly denoted latin1 encoding, which is slightly misleading as ISO Latin-1 is a character code range, not an encoding.

UTF-8 - Each character is stored in one to four bytes depending on code point. The encoding is backward compatible with bytewise representation of 7-bit ASCII, as all 7-bit characters are stored in one single byte in UTF-8. The characters beyond code point 127 are stored in more bytes, letting the most significant bit in the first character indicate a multi-byte character. For details on the encoding, the RFC is publicly available.

Notice that UTF-8 is not compatible with bytewise representation for code points from 128 through 255, so an ISO Latin-1 bytewise representation is generally incompatible with UTF-8.

UTF-16 - This encoding has many similarities to UTF-8, but the basic unit is a 16-bit number. This means that all characters occupy at least two bytes, and some high numbers four bytes. Some programs, libraries, and operating systems claiming to use UTF-16 only allow for characters that can be stored in one 16-bit entity, which is usually sufficient to handle living languages. As the basic unit is more than one byte, byte-order issues occur, which is why UTF-16 exists in both a big-endian and a little-endian variant.

In Erlang, the full UTF-16 range is supported when applicable, like in the unicode module and in the bit syntax.

UTF-32 - The most straightforward representation. Each character is stored in one single 32-bit number. There is no need for escapes or any variable number of entities for one character. All Unicode code points can be stored in one single 32-bit entity. As with UTF-16, there are byte-order issues. UTF-32 can be both big-endian and little-endian.

UCS-4 - Basically the same as UTF-32, but without some Unicode semantics, defined by IEEE, and has little use as a separate encoding standard. For all normal (and possibly abnormal) use, UTF-32 and UCS-4 are interchangeable.

Certain number ranges are unused in the Unicode standard and certain ranges are even deemed invalid. The most notable invalid range is 16#D800-16#DFFF, as the UTF-16 encoding does not allow for encoding of these numbers. This is possibly because the UTF-16 encoding standard, from the beginning, was expected to be able to hold all Unicode characters in one 16-bit entity, but was then extended, leaving a hole in the Unicode range to handle backward compatibility.

Code point 16#FEFF is used for Byte Order Marks (BOMs) and use of that character is not encouraged in other contexts. It is valid though, as the character "ZWNBS" (Zero Width Non Breaking Space). BOMs are used to identify encodings and byte order for programs where such parameters are not known in advance. BOMs are more seldom used than expected, but can become more widely spread as they provide the means for programs to make educated guesses about the Unicode format of a certain file.

Areas of Unicode Support

To support Unicode in Erlang, problems in various areas have been addressed. This section describes each area briefly and more thoroughly later in this User's Guide.

Representation - To handle Unicode characters in Erlang, a common representation in both lists and binaries is needed. EEP (10) and the subsequent initial implementation in Erlang/OTP R13A settled a standard representation of Unicode characters in Erlang.

Manipulation - The Unicode characters need to be processed by the Erlang program, which is why library functions must be able to handle them. In some cases functionality has been added to already existing interfaces (as the string module now can handle strings with any code points). In some cases new functionality or options have been added (as in the io module, the file handling, the unicode module, and the bit syntax). Today most modules in Kernel and STDLIB, as well as the VM are Unicode-aware.

File I/O - I/O is by far the most problematic area for Unicode. A file is an entity where bytes are stored, and the lore of programming has been to treat characters and bytes as interchangeable. With Unicode characters, you must decide on an encoding when you want to store the data in a file. In Erlang, you can open a text file with an encoding option, so that you can read characters from it rather than bytes, but you can also open a file for bytewise I/O.

The Erlang I/O-system has been designed (or at least used) in a way where you expect any I/O server to handle any string data. That is, however, no longer the case when working with Unicode characters. The Erlang programmer must now know the capabilities of the device where the data ends up. Also, ports in Erlang are byte-oriented, so an arbitrary string of (Unicode) characters cannot be sent to a port without first converting it to an encoding of choice.

Terminal I/O - Terminal I/O is slightly easier than file I/O. The output is meant for human reading and is usually Erlang syntax (for example, in the shell). There exists syntactic representation of any Unicode character without displaying the glyph (instead written as \x{HHH}). Unicode data can therefore usually be displayed even if the terminal as such does not support the whole Unicode range.

Filenames - Filenames can be stored as Unicode strings in different ways depending on the underlying operating system and file system. This can be handled fairly easy by a program. The problems arise when the file system is inconsistent in its encodings. For example, Linux allows files to be named with any sequence of bytes, leaving to each program to interpret those bytes. On systems where these "transparent" filenames are used, Erlang must be informed about the filename encoding by a startup flag. The default is bytewise interpretation, which is usually wrong, but allows for interpretation of all filenames.

The concept of "raw filenames" can be used to handle wrongly encoded filenames if one enables Unicode filename translation (+fnu) on platforms where this is not the default.

Source code encoding - The Erlang source code has support for the UTF-8 encoding and bytewise encoding. The default in Erlang/OTP R16B was bytewise (latin1) encoding. It was changed to UTF-8 in Erlang/OTP 17.0. You can control the encoding by a comment like the following in the beginning of the file:

%% -*- coding: utf-8 -*-

This of course requires your editor to support UTF-8 as well. The same comment is also interpreted by functions like file:consult/1, the release handler, and so on, so that you can have all text files in your source directories in UTF-8 encoding.

The language - Having the source code in UTF-8 also allows you to write string literals, function names, and atoms containing Unicode characters with code points > 255. Module names, application names, and node names are still restricted to the ISO Latin-1 range. Binary literals, where you use type /utf8, can also be expressed using Unicode characters > 255. Having module names or application names using characters other than 7-bit ASCII can cause trouble on operating systems with inconsistent file naming schemes, and can hurt portability, so it is not recommended.

EEP 40 suggests that the language is also to allow for Unicode characters > 255 in variable names. Whether to implement that EEP is yet to be decided.

Standard Unicode Representation

In Erlang, strings are lists of integers. A string was until Erlang/OTP R13 defined to be encoded in the ISO Latin-1 (ISO 8859-1) character set, which is, code point by code point, a subrange of the Unicode character set.

The standard list encoding for strings was therefore easily extended to handle the whole Unicode range. A Unicode string in Erlang is a list containing integers, where each integer is a valid Unicode code point and represents one character in the Unicode character set.

Erlang strings in ISO Latin-1 are a subset of Unicode strings.

Only if a string contains code points < 256, can it be directly converted to a binary by using, for example, erlang:iolist_to_binary/1 or can be sent directly to a port. If the string contains Unicode characters > 255, an encoding must be decided upon and the string is to be converted to a binary in the preferred encoding using unicode:characters_to_binary/1,2,3. Strings are not generally lists of bytes, as they were before Erlang/OTP R13, they are lists of characters. Characters are not generally bytes, they are Unicode code points.

Binaries are more troublesome. For performance reasons, programs often store textual data in binaries instead of lists, mainly because they are more compact (one byte per character instead of two words per character, as is the case with lists). Using erlang:list_to_binary/1, an ISO Latin-1 Erlang string can be converted into a binary, effectively using bytewise encoding: one byte per character. This was convenient for those limited Erlang strings, but cannot be done for arbitrary Unicode lists.

As the UTF-8 encoding is widely spread and provides some backward compatibility in the 7-bit ASCII range, it is selected as the standard encoding for Unicode characters in binaries for Erlang.

The standard binary encoding is used whenever a library function in Erlang is to handle Unicode data in binaries, but is of course not enforced when communicating externally. Functions and bit syntax exist to encode and decode both UTF-8, UTF-16, and UTF-32 in binaries. However, library functions dealing with binaries and Unicode in general only deal with the default encoding.

Character data can be combined from many sources, sometimes available in a mix of strings and binaries. Erlang has for long had the concept of iodata or iolists, where binaries and lists can be combined to represent a sequence of bytes. In the same way, the Unicode-aware modules often allow for combinations of binaries and lists, where the binaries have characters encoded in UTF-8 and the lists contain such binaries or numbers representing Unicode code points:

unicode_binary() = binary() with characters encoded in UTF-8 coding standard

chardata() = charlist() | unicode_binary()

charlist() = maybe_improper_list(char() | unicode_binary() | charlist(),
  unicode_binary() | nil())

The module unicode even supports similar mixes with binaries containing other encodings than UTF-8, but that is a special case to allow for conversions to and from external data:

external_unicode_binary() = binary() with characters coded in a user-specified
  Unicode encoding other than UTF-8 (UTF-16 or UTF-32)

external_chardata() = external_charlist() | external_unicode_binary()

external_charlist() = maybe_improper_list(char() | external_unicode_binary() |
  external_charlist(), external_unicode_binary() | nil())

Basic Language Support

As from Erlang/OTP R16, Erlang source files can be written in UTF-8 or bytewise (latin1) encoding. For information about how to state the encoding of an Erlang source file, see the epp module. As from Erlang/OTP R16, strings and comments can be written using Unicode. As from Erlang/OTP 20, also atoms and functions can be written using Unicode. Modules, applications, and nodes must still be named using characters from the ISO Latin-1 character set. (These restrictions in the language are independent of the encoding of the source file.)

Bit Syntax

The bit syntax contains types for handling binary data in the three main encodings. The types are named utf8, utf16, and utf32. The utf16 and utf32 types can be in a big-endian or a little-endian variant:

<<Ch/utf8,_/binary>> = Bin1,
<<Ch/utf16-little,_/binary>> = Bin2,
Bin3 = <<$H/utf32-little, $e/utf32-little, $l/utf32-little, $l/utf32-little,
$o/utf32-little>>,

For convenience, literal strings can be encoded with a Unicode encoding in binaries using the following (or similar) syntax:

Bin4 = <<"Hello"/utf16>>,

String and Character Literals

For source code, there is an extension to syntax \OOO (backslash followed by three octal numbers) and \xHH (backslash followed by x, followed by two hexadecimal characters), namely \x{H ...} (backslash followed by x, followed by left curly bracket, any number of hexadecimal digits, and a terminating right curly bracket). This allows for entering characters of any code point literally in a string even when the encoding of the source file is bytewise (latin1).

In the shell, if using a Unicode input device, or in source code stored in UTF-8, $ can be followed directly by a Unicode character producing an integer. In the following example, the code point of a Cyrillic с is output:

7> $с.
1089

Heuristic String Detection

In certain output functions and in the output of return values in the shell, Erlang tries to detect string data in lists and binaries heuristically. Typically you will see heuristic detection in a situation like this:

1> [97,98,99].
"abc"
2> <<97,98,99>>.
<<"abc">>
3> <<195,165,195,164,195,182>>.
<<"åäö"/utf8>>

Here the shell detects lists containing printable characters or binaries containing printable characters in bytewise or UTF-8 encoding. But what is a printable character? One view is that anything the Unicode standard thinks is printable, is also printable according to the heuristic detection. The result is then that almost any list of integers are deemed a string, and all sorts of characters are printed, maybe also characters that your terminal lacks in its font set (resulting in some unappreciated generic output). Another way is to keep it backward compatible so that only the ISO Latin-1 character set is used to detect a string. A third way is to let the user decide exactly what Unicode ranges that are to be viewed as characters.

As from Erlang/OTP R16B you can select the ISO Latin-1 range or the whole Unicode range by supplying startup flag +pc latin1 or +pc unicode, respectively. For backward compatibility, latin1 is default. This only controls how heuristic string detection is done. More ranges are expected to be added in the future, enabling tailoring of the heuristics to the language and region relevant to the user.

The following examples show the two startup options:

$ erl +pc latin1
Erlang R16B (erts-5.10.1) [source] [async-threads:0] [hipe] [kernel-poll:false]

Eshell V5.10.1  (abort with ^G)
1> [1024].
[1024]
2> [1070,1085,1080,1082,1086,1076].
[1070,1085,1080,1082,1086,1076]
3> [229,228,246].
"åäö"
4> <<208,174,208,189,208,184,208,186,208,190,208,180>>.
<<208,174,208,189,208,184,208,186,208,190,208,180>>
5> <<229/utf8,228/utf8,246/utf8>>.
<<"åäö"/utf8>>

$ erl +pc unicode
Erlang R16B (erts-5.10.1) [source] [async-threads:0] [hipe] [kernel-poll:false]

Eshell V5.10.1  (abort with ^G)
1> [1024].
"Ѐ"
2> [1070,1085,1080,1082,1086,1076].
"Юникод"
3> [229,228,246].
"åäö"
4> <<208,174,208,189,208,184,208,186,208,190,208,180>>.
<<"Юникод"/utf8>>
5> <<229/utf8,228/utf8,246/utf8>>.
<<"åäö"/utf8>>

In the examples, you can see that the default Erlang shell interprets only characters from the ISO Latin1 range as printable and only detects lists or binaries with those "printable" characters as containing string data. The valid UTF-8 binary containing the Russian word "Юникод", is not printed as a string. When started with all Unicode characters printable (+pc unicode), the shell outputs anything containing printable Unicode data (in binaries, either UTF-8 or bytewise encoded) as string data.

These heuristics are also used by io:format/2, io_lib:format/2, and friends when modifier t is used with ~p or ~P:

$ erl +pc latin1
Erlang R16B (erts-5.10.1) [source] [async-threads:0] [hipe] [kernel-poll:false]

Eshell V5.10.1  (abort with ^G)
1> io:format("~tp~n",[{<<"åäö">>, <<"åäö"/utf8>>, <<208,174,208,189,208,184,208,186,208,190,208,180>>}]).
{<<"åäö">>,<<"åäö"/utf8>>,<<208,174,208,189,208,184,208,186,208,190,208,180>>}
ok

$ erl +pc unicode
Erlang R16B (erts-5.10.1) [source] [async-threads:0] [hipe] [kernel-poll:false]

Eshell V5.10.1  (abort with ^G)
1> io:format("~tp~n",[{<<"åäö">>, <<"åäö"/utf8>>, <<208,174,208,189,208,184,208,186,208,190,208,180>>}]).
{<<"åäö">>,<<"åäö"/utf8>>,<<"Юникод"/utf8>>}
ok

Notice that this only affects heuristic interpretation of lists and binaries on output. For example, the ~ts format sequence always outputs a valid list of characters, regardless of the +pc setting, as the programmer has explicitly requested string output.

The Interactive Shell

The interactive Erlang shell can support Unicode input and output.

On Windows, proper operation requires that a suitable font is installed and selected for the Erlang application to use. If no suitable font is available on your system, try installing the DejaVu fonts, which are freely available, and then select that font in the Erlang shell application.

On Unix-like operating systems, the terminal is to be able to handle UTF-8 on input and output (this is done by, for example, modern versions of XTerm, KDE Konsole, and the Gnome terminal) and your locale settings must be proper. As an example, a LANG environment variable can be set as follows:

$ echo $LANG
en_US.UTF-8

Most systems handle variable LC_CTYPE before LANG, so if that is set, it must be set to UTF-8:

$ echo $LC_CTYPE
en_US.UTF-8

The LANG or LC_CTYPE setting are to be consistent with what the terminal is capable of. There is no portable way for Erlang to ask the terminal about its UTF-8 capacity, we have to rely on the language and character type settings.

To investigate what Erlang thinks about the terminal, the call io:getopts() can be used when the shell is started:

$ LC_CTYPE=en_US.ISO-8859-1 erl
Erlang R16B (erts-5.10.1) [source] [async-threads:0] [hipe] [kernel-poll:false]

Eshell V5.10.1  (abort with ^G)
1> lists:keyfind(encoding, 1, io:getopts()).
{encoding,latin1}
2> q().
ok
$ LC_CTYPE=en_US.UTF-8 erl
Erlang R16B (erts-5.10.1) [source] [async-threads:0] [hipe] [kernel-poll:false]

Eshell V5.10.1  (abort with ^G)
1> lists:keyfind(encoding, 1, io:getopts()).
{encoding,unicode}
2>

When (finally?) everything is in order with the locale settings, fonts. and the terminal emulator, you have probably found a way to input characters in the script you desire. For testing, the simplest way is to add some keyboard mappings for other languages, usually done with some applet in your desktop environment.

In a KDE environment, select KDE Control Center (Personal Settings) > Regional and Accessibility > Keyboard Layout.

On Windows XP, select Control Panel > Regional and Language Options, select tab Language, and click button Details... in the square named Text Services and Input Languages.

Your environment probably provides similar means of changing the keyboard layout. Ensure that you have a way to switch back and forth between keyboards easily if you are not used to this. For example, entering commands using a Cyrillic character set is not easily done in the Erlang shell.

Now you are set up for some Unicode input and output. The simplest thing to do is to enter a string in the shell:

$ erl
Erlang R16B (erts-5.10.1) [source] [async-threads:0] [hipe] [kernel-poll:false]

Eshell V5.10.1  (abort with ^G)
1> lists:keyfind(encoding, 1, io:getopts()).
{encoding,unicode}
2> "Юникод".
"Юникод"
3> io:format("~ts~n", [v(2)]).
Юникод
ok
4>

While strings can be input as Unicode characters, the language elements are still limited to the ISO Latin-1 character set. Only character constants and strings are allowed to be beyond that range:

$ erl
Erlang R16B (erts-5.10.1) [source] [async-threads:0] [hipe] [kernel-poll:false]

Eshell V5.10.1  (abort with ^G)
1> $ξ.
958
2> Юникод.
* 1: illegal character
2>

Escripts and non-interactive I/O

When Erlang is started without an interactive shell (-noshell, -noinput or as an escript) the unicode support is identified using environment variables just as for interactive shells. Working with unicode in non-interactive sessions works just the same as for interactive sessions.

In some situations you may need to be able to read and write raw bytes from standard_io. If that is the case, then you want to set the standard_io_encoding configuration parameter to latin1 and use the file API to read and write data (as explained in Unicode Data in Files).

In the example below we first read the character ξ from standard_io and then print the charlist() represented by it.

#!/usr/bin/env escript
%%! -kernel standard_io_encoding latin1

main(_) ->
  {ok, Char} = file:read_line(standard_io),
  ok = file:write(standard_io, string:trim(Char)),
  ok = file:write(standard_io, io_lib:format(": ~w~n",[string:trim(Char)])),
  ok.

$ escript test.es
ξ
ξ: [206,190]

ξ would normally be represented as the integer 958, but since we are using bytewise encoding (latin1), it is represented by 206 and 190, which is the utf-8 bytes representing ξ. When we echo those bytes back to standard_io, the terminal will see the bytes as utf-8 and show the correct value even though in Erlang we never knew that it was indeed a unicode string.

Unicode Filenames

Most modern operating systems support Unicode filenames in some way. There are many different ways to do this and Erlang by default treats the different approaches differently:

Mandatory Unicode file naming - Windows, Android and, for most cases, MacOS X enforce Unicode support for filenames. All files created in the file system have names that can consistently be interpreted. In MacOS X and Android, all filenames are retrieved in UTF-8 encoding. In Windows, each system call handling filenames has a special Unicode-aware variant, giving much the same effect. There are no filenames on these systems that are not Unicode filenames. So, the default behavior of the Erlang VM is to work in "Unicode filename translation mode". This means that a filename can be specified as a Unicode list, which is automatically translated to the proper name encoding for the underlying operating system and file system.

Doing, for example, a file:list_dir/1 on one of these systems can return Unicode lists with code points > 255, depending on the content of the file system.

Transparent file naming - Most Unix operating systems have adopted a simpler approach, namely that Unicode file naming is not enforced, but by convention. Those systems usually use UTF-8 encoding for Unicode filenames, but do not enforce it. On such a system, a filename containing characters with code points from 128 through 255 can be named as plain ISO Latin-1 or use UTF-8 encoding. As no consistency is enforced, the Erlang VM cannot do consistent translation of all filenames.

By default on such systems, Erlang starts in utf8 filename mode if the terminal supports UTF-8, otherwise in latin1 mode.

In latin1 mode, filenames are bytewise encoded. This allows for list representation of all filenames in the system. However, a a file named "Östersund.txt", appears in file:list_dir/1 either as "Östersund.txt" (if the filename was encoded in bytewise ISO Latin-1 by the program creating the file) or more probably as [195,150,115,116,101,114,115,117,110,100], which is a list containing UTF-8 bytes (not what you want). If you use Unicode filename translation on such a system, non-UTF-8 filenames are ignored by functions like file:list_dir/1. They can be retrieved with function file:list_dir_all/1, but wrongly encoded filenames appear as "raw filenames".

The Unicode file naming support was introduced in Erlang/OTP R14B01. A VM operating in Unicode filename translation mode can work with files having names in any language or character set (as long as it is supported by the underlying operating system and file system). The Unicode character list is used to denote filenames or directory names. If the file system content is listed, you also get Unicode lists as return value. The support lies in the Kernel and STDLIB modules, which is why most applications (that do not explicitly require the filenames to be in the ISO Latin-1 range) benefit from the Unicode support without change.

On operating systems with mandatory Unicode filenames, this means that you more easily conform to the filenames of other (non-Erlang) applications. You can also process filenames that, at least on Windows, were inaccessible (because of having names that could not be represented in ISO Latin-1). Also, you avoid creating incomprehensible filenames on MacOS X, as the vfs layer of the operating system accepts all your filenames as UTF-8 does not rewrite them.

For most systems, turning on Unicode filename translation is no problem even if it uses transparent file naming. Very few systems have mixed filename encodings. A consistent UTF-8 named system works perfectly in Unicode filename mode. It was still, however, considered experimental in Erlang/OTP R14B01 and is still not the default on such systems.

Unicode filename translation is turned on with switch +fnu. On Linux, a VM started without explicitly stating the filename translation mode defaults to latin1 as the native filename encoding. On Windows, MacOS X and Android, the default behavior is that of Unicode filename translation. Therefore file:native_name_encoding/0 by default returns utf8 on those systems (Windows does not use UTF-8 on the file system level, but this can safely be ignored by the Erlang programmer). The default behavior can, as stated earlier, be changed using option +fnu or +fnl to the VM, see the erl program. If the VM is started in Unicode filename translation mode, file:native_name_encoding/0 returns atom utf8. Switch +fnu can be followed by w, i, or e to control how wrongly encoded filenames are to be reported.

w means that a warning is sent to the error_logger whenever a wrongly encoded filename is "skipped" in directory listings. w is the default.

i means that wrongly encoded filenames are silently ignored.

e means that the API function returns an error whenever a wrongly encoded filename (or directory name) is encountered.

Notice that file:read_link/1 always returns an error if the link points to an invalid filename.

In Unicode filename mode, filenames given to BIF open_port/2 with option {spawn_executable,...} are also interpreted as Unicode. So is the parameter list specified in option args available when using spawn_executable. The UTF-8 translation of arguments can be avoided using binaries, see section Notes About Raw Filenames.

Notice that the file encoding options specified when opening a file has nothing to do with the filename encoding convention. You can very well open files containing data encoded in UTF-8, but having filenames in bytewise (latin1) encoding or conversely.

Note

Erlang drivers and NIF-shared objects still cannot be named with names containing code points > 127. This limitation will be removed in a future release. However, Erlang modules can, but it is definitely not a good idea and is still considered experimental.

Notes About Raw Filenames

Note

Note that raw filenames not necessarily are encoded the same way as on the OS level.

Raw filenames were introduced together with Unicode filename support in ERTS 5.8.2 (Erlang/OTP R14B01). The reason "raw filenames" were introduced in the system was to be able to represent filenames, specified in different encodings on the same system, consistently. It can seem practical to have the VM automatically translate a filename that is not in UTF-8 to a list of Unicode characters, but this would open up for both duplicate filenames and other inconsistent behavior.

Consider a directory containing a file named "björn" in ISO Latin-1, while the Erlang VM is operating in Unicode filename mode (and therefore expects UTF-8 file naming). The ISO Latin-1 name is not valid UTF-8 and one can be tempted to think that automatic conversion in, for example, file:list_dir/1 is a good idea. But what would happen if we later tried to open the file and have the name as a Unicode list (magically converted from the ISO Latin-1 filename)? The VM converts the filename to UTF-8, as this is the encoding expected. Effectively this means trying to open the file named <<"björn"/utf8>>. This file does not exist, and even if it existed it would not be the same file as the one that was listed. We could even create two files named "björn", one named in UTF-8 encoding and one not. If file:list_dir/1 would automatically convert the ISO Latin-1 filename to a list, we would get two identical filenames as the result. To avoid this, we must differentiate between filenames that are properly encoded according to the Unicode file naming convention (that is, UTF-8) and filenames that are invalid under the encoding. By the common function file:list_dir/1, the wrongly encoded filenames are ignored in Unicode filename translation mode, but by function file:list_dir_all/1 the filenames with invalid encoding are returned as "raw" filenames, that is, as binaries.

The file module accepts raw filenames as input. open_port({spawn_executable, ...} ...) also accepts them. As mentioned earlier, the arguments specified in the option list to open_port({spawn_executable, ...} ...) undergo the same conversion as the filenames, meaning that the executable is provided with arguments in UTF-8 as well. This translation is avoided consistently with how the filenames are treated, by giving the argument as a binary.

To force Unicode filename translation mode on systems where this is not the default was considered experimental in Erlang/OTP R14B01. This was because the initial implementation did not ignore wrongly encoded filenames, so that raw filenames could spread unexpectedly throughout the system. As from Erlang/OTP R16B, the wrongly encoded filenames are only retrieved by special functions (such as file:list_dir_all/1). Since the impact on existing code is therefore much lower it is now supported. Unicode filename translation is expected to be default in future releases.

Even if you are operating without Unicode file naming translation automatically done by the VM, you can access and create files with names in UTF-8 encoding by using raw filenames encoded as UTF-8. Enforcing the UTF-8 encoding regardless of the mode the Erlang VM is started in can in some circumstances be a good idea, as the convention of using UTF-8 filenames is spreading.

Notes About MacOS X

The vfs layer of MacOS X enforces UTF-8 filenames in an aggressive way. Older versions did this by refusing to create non-UTF-8 conforming filenames, while newer versions replace offending bytes with the sequence "%HH", where HH is the original character in hexadecimal notation. As Unicode translation is enabled by default on MacOS X, the only way to come up against this is to either start the VM with flag +fnl or to use a raw filename in bytewise (latin1) encoding. If using a raw filename, with a bytewise encoding containing characters from 127 through 255, to create a file, the file cannot be opened using the same name as the one used to create it. There is no remedy for this behavior, except keeping the filenames in the correct encoding.

MacOS X reorganizes the filenames so that the representation of accents, and so on, uses the "combining characters". For example, character ö is represented as code points [111,776], where 111 is character o and 776 is the special accent character "Combining Diaeresis". This way of normalizing Unicode is otherwise very seldom used. Erlang normalizes those filenames in the opposite way upon retrieval, so that filenames using combining accents are not passed up to the Erlang application. In Erlang, filename "björn" is retrieved as [98,106,246,114,110], not as [98,106,117,776,114,110], although the file system can think differently. The normalization into combining accents is redone when accessing files, so this can usually be ignored by the Erlang programmer.

Unicode in Environment and Parameters

Environment variables and their interpretation are handled much in the same way as filenames. If Unicode filenames are enabled, environment variables as well as parameters to the Erlang VM are expected to be in Unicode.

If Unicode filenames are enabled, the calls to os:getenv/0,1, os:putenv/2, and os:unsetenv/1 handle Unicode strings. On Unix-like platforms, the built-in functions translate environment variables in UTF-8 to/from Unicode strings, possibly with code points > 255. On Windows, the Unicode versions of the environment system API are used, and code points > 255 are allowed.

On Unix-like operating systems, parameters are expected to be UTF-8 without translation if Unicode filenames are enabled.

Unicode-Aware Modules

Most of the modules in Erlang/OTP are Unicode-unaware in the sense that they have no notion of Unicode and should not have. Typically they handle non-textual or byte-oriented data (such as gen_tcp).

Modules handling textual data (such as io_lib and string are sometimes subject to conversion or extension to be able to handle Unicode characters.

Fortunately, most textual data has been stored in lists and range checking has been sparse, so modules like string work well for Unicode strings with little need for conversion or extension.

Some modules are, however, changed to be explicitly Unicode-aware. These modules include:

unicode - The unicode module is clearly Unicode-aware. It contains functions for conversion between different Unicode formats and some utilities for identifying byte order marks. Few programs handling Unicode data survive without this module.

io - The io module has been extended along with the actual I/O protocol to handle Unicode data. This means that many functions require binaries to be in UTF-8, and there are modifiers to format control sequences to allow for output of Unicode strings.

file, group, user - I/O-servers throughout the system can handle Unicode data and have options for converting data upon output or input to/from the device. As shown earlier, the shell module has support for Unicode terminals and the file module allows for translation to and from various Unicode formats on disk.

Reading and writing of files with Unicode data is, however, not best done with the file module, as its interface is byte-oriented. A file opened with a Unicode encoding (like UTF-8) is best read or written using the io module.

re - The re module allows for matching Unicode strings as a special option. As the library is centered on matching in binaries, the Unicode support is UTF-8-centered.

wx - The graphical library wx has extensive support for Unicode text.

The string module works perfectly for Unicode strings and ISO Latin-1 strings, except the language-dependent functions string:uppercase/1 and string:lowercase/1. These two functions can never function correctly for Unicode characters in their current form, as there are language and locale issues to consider when converting text between cases. Converting case in an international environment is a large subject not yet addressed in OTP.

Unicode Data in Files

Although Erlang can handle Unicode data in many forms does not automatically mean that the content of any file can be Unicode text. The external entities, such as ports and I/O servers, are not generally Unicode capable.

Ports are always byte-oriented, so before sending data that you are not sure is bytewise-encoded to a port, ensure to encode it in a proper Unicode encoding. Sometimes this means that only part of the data must be encoded as, for example, UTF-8. Some parts can be binary data (like a length indicator) or something else that must not undergo character encoding, so no automatic translation is present.

I/O servers behave a little differently. The I/O servers connected to terminals (or stdout) can usually cope with Unicode data regardless of the encoding option. This is convenient when one expects a modern environment but do not want to crash when writing to an archaic terminal or pipe.

A file can have an encoding option that makes it generally usable by the io module (for example {encoding,utf8}), but is by default opened as a byte-oriented file. The file module is byte-oriented, so only ISO Latin-1 characters can be written using that module. Use the io module if Unicode data is to be output to a file with other encoding than latin1 (bytewise encoding). It is slightly confusing that a file opened with, for example, file:open(Name,[read,{encoding,utf8}]) cannot be properly read using file:read(File,N), but using the io module to retrieve the Unicode data from it. The reason is that file:read and file:write (and friends) are purely byte-oriented, and should be, as that is the way to access files other than text files, byte by byte. As with ports, you can write encoded data into a file by "manually" converting the data to the encoding of choice (using the unicode module or the bit syntax) and then output it on a bytewise (latin1) encoded file.

Recommendations:

Use the file module for files opened for bytewise access ({encoding,latin1}).

Use the io module when accessing files with any other encoding (for example {encoding,utf8}).

Functions reading Erlang syntax from files recognize the coding: comment and can therefore handle Unicode data on input. When writing Erlang terms to a file, you are advised to insert such comments when applicable:

$ erl +fna +pc unicode
Erlang R16B (erts-5.10.1) [source]  [async-threads:0] [hipe] [kernel-poll:false]

Eshell V5.10.1  (abort with ^G)
1> file:write_file("test.term",<<"%% coding: utf-8\n[{\"Юникод\",4711}].\n"/utf8>>).
ok
2> file:consult("test.term").
{ok,[[{"Юникод",4711}]]}

Summary of Options

The Unicode support is controlled by both command-line switches, some standard environment variables, and the OTP version you are using. Most options affect mainly how Unicode data is displayed, not the functionality of the APIs in the standard libraries. This means that Erlang programs usually do not need to concern themselves with these options, they are more for the development environment. An Erlang program can be written so that it works well regardless of the type of system or the Unicode options that are in effect.

Here follows a summary of the settings affecting Unicode:

The LANG and LC_CTYPE environment variables - The language setting in the operating system mainly affects the shell. The terminal (that is, the group leader) operates with {encoding, unicode} only if the environment tells it that UTF-8 is allowed. This setting is to correspond to the terminal you are using.

The environment can also affect filename interpretation, if Erlang is started with flag +fna (which is default from Erlang/OTP 17.0).

You can check the setting of this by calling io:getopts(), which gives you an option list containing {encoding,unicode} or {encoding,latin1}.

The +pc {unicode|latin1} flag to erl(1) - This flag affects what is interpreted as string data when doing heuristic string detection in the shell and in io/ io_lib:format with the "~tp" and ~tP formatting instructions, as described earlier.

You can check this option by calling io:printable_range/0, which returns unicode or latin1. To be compatible with future (expected) extensions to the settings, rather use io_lib:printable_list/1 to check if a list is printable according to the setting. That function takes into account new possible settings returned from io:printable_range/0.

The +fn{l|u|a} [{w|i|e}] flag to erl(1) - This flag affects how the filenames are to be interpreted. On operating systems with transparent file naming, this must be specified to allow for file naming in Unicode characters (and for correct interpretation of filenames containing characters > 255).

+fnl means bytewise interpretation of filenames, which was the usual way to represent ISO Latin-1 filenames before UTF-8 file naming got widespread.

+fnu means that filenames are encoded in UTF-8, which is nowadays the common scheme (although not enforced).

+fna means that you automatically select between +fnl and +fnu, based on environment variables LANG and LC_CTYPE. This is optimistic heuristics indeed, nothing enforces a user to have a terminal with the same encoding as the file system, but this is usually the case. This is the default on all Unix-like operating systems, except MacOS X.

The filename translation mode can be read with function file:native_name_encoding/0, which returns latin1 (bytewise encoding) or utf8.

epp:default_encoding/0 - This function returns the default encoding for Erlang source files (if no encoding comment is present) in the currently running release. In Erlang/OTP R16B, latin1 (bytewise encoding) was returned. As from Erlang/OTP 17.0, utf8 is returned.

The encoding of each file can be specified using comments as described in the epp module.

io:setopts/1,2 and standard_io_encoding - When Erlang is started the encoding for standard_io is by default set to what the locale settings indicate. You can override the default by setting the kernel configuration parameter standard_io_encoding to the desired encoding.

You can set the encoding of a file or other I/O server with function io:setopts/2. This can also be set when opening a file. Setting the terminal (or other standard_io server) unconditionally to option {encoding,utf8} implies that UTF-8 encoded characters are written to the device, regardless of how Erlang was started or the user's environment.

Note

If you use io:setopts/2 to change the encoding of standard_io the I/O server may already have read some data using the default encoding. To avoid this you should set the encoding using standard_io_encoding.

Opening files with option encoding is convenient when writing or reading text files in a known encoding.

You can retrieve the encoding setting for an I/O server with function io:getopts().

Recipes

When starting with Unicode, one often stumbles over some common issues. This section describes some methods of dealing with Unicode data.

Byte Order Marks

A common method of identifying encoding in text files is to put a Byte Order Mark (BOM) first in the file. The BOM is the code point 16#FEFF encoded in the same way as the remaining file. If such a file is to be read, the first few bytes (depending on encoding) are not part of the text. This code outlines how to open a file that is believed to have a BOM, and sets the files encoding and position for further sequential reading (preferably using the io module).

Notice that error handling is omitted from the code:

open_bom_file_for_reading(File) ->
    {ok,F} = file:open(File,[read,binary]),
    {ok,Bin} = file:read(F,4),
    {Type,Bytes} = unicode:bom_to_encoding(Bin),
    file:position(F,Bytes),
    io:setopts(F,[{encoding,Type}]),
    {ok,F}.

Function unicode:bom_to_encoding/1 identifies the encoding from a binary of at least four bytes. It returns, along with a term suitable for setting the encoding of the file, the byte length of the BOM, so that the file position can be set accordingly. Notice that function file:position/2 always works on byte-offsets, so that the byte length of the BOM is needed.

To open a file for writing and place the BOM first is even simpler:

open_bom_file_for_writing(File,Encoding) ->
    {ok,F} = file:open(File,[write,binary]),
    ok = file:write(File,unicode:encoding_to_bom(Encoding)),
    io:setopts(F,[{encoding,Encoding}]),
    {ok,F}.

The file is in both these cases then best processed using the io module, as the functions in that module can handle code points beyond the ISO Latin-1 range.

Formatted I/O

When reading and writing to Unicode-aware entities, like a file opened for Unicode translation, you probably want to format text strings using the functions in the io module or the io_lib module. For backward compatibility reasons, these functions do not accept any list as a string, but require a special translation modifier when working with Unicode texts. The modifier is t. When applied to control character s in a formatting string, it accepts all Unicode code points and expects binaries to be in UTF-8:

1> io:format("~ts~n",[<<"åäö"/utf8>>]).
åäö
ok
2> io:format("~s~n",[<<"åäö"/utf8>>]).
Ã¥Ã¤Ã¶
ok

Clearly, the second io:format/2 gives undesired output, as the UTF-8 binary is not in latin1. For backward compatibility, the non-prefixed control character s expects bytewise-encoded ISO Latin-1 characters in binaries and lists containing only code points < 256.

As long as the data is always lists, modifier t can be used for any string, but when binary data is involved, care must be taken to make the correct choice of formatting characters. A bytewise-encoded binary is also interpreted as a string, and printed even when using ~ts, but it can be mistaken for a valid UTF-8 string. Avoid therefore using the ~ts control if the binary contains bytewise-encoded characters and not UTF-8.

Function io_lib:format/2 behaves similarly. It is defined to return a deep list of characters and the output can easily be converted to binary data for outputting on any device by a simple erlang:list_to_binary/1. When the translation modifier is used, the list can, however, contain characters that cannot be stored in one byte. The call to erlang:list_to_binary/1 then fails. However, if the I/O server you want to communicate with is Unicode-aware, the returned list can still be used directly:

$ erl +pc unicode
Erlang R16B (erts-5.10.1) [source] [async-threads:0] [hipe] [kernel-poll:false]

Eshell V5.10.1 (abort with ^G)
1> io_lib:format("~ts~n", ["Γιούνικοντ"]).
["Γιούνικοντ","\n"]
2> io:put_chars(io_lib:format("~ts~n", ["Γιούνικοντ"])).
Γιούνικοντ
ok

The Unicode string is returned as a Unicode list, which is recognized as such, as the Erlang shell uses the Unicode encoding (and is started with all Unicode characters considered printable). The Unicode list is valid input to function io:put_chars/2, so data can be output on any Unicode-capable device. If the device is a terminal, characters are output in format \x{H...} if encoding is latin1. Otherwise in UTF-8 (for the non-interactive terminal: "oldshell" or "noshell") or whatever is suitable to show the character properly (for an interactive terminal: the regular shell).

So, you can always send Unicode data to the standard_io device. Files, however, accept only Unicode code points beyond ISO Latin-1 if encoding is set to something else than latin1.

Heuristic Identification of UTF-8

While it is strongly encouraged that the encoding of characters in binary data is known before processing, that is not always possible. On a typical Linux system, there is a mix of UTF-8 and ISO Latin-1 text files, and there are seldom any BOMs in the files to identify them.

UTF-8 is designed so that ISO Latin-1 characters with numbers beyond the 7-bit ASCII range are seldom considered valid when decoded as UTF-8. Therefore one can usually use heuristics to determine if a file is in UTF-8 or if it is encoded in ISO Latin-1 (one byte per character). The unicode module can be used to determine if data can be interpreted as UTF-8:

heuristic_encoding_bin(Bin) when is_binary(Bin) ->
    case unicode:characters_to_binary(Bin,utf8,utf8) of
	Bin ->
	    utf8;
	_ ->
	    latin1
    end.

If you do not have a complete binary of the file content, you can instead chunk through the file and check part by part. The return-tuple {incomplete,Decoded,Rest} from function unicode:characters_to_binary/1,2,3 comes in handy. The incomplete rest from one chunk of data read from the file is prepended to the next chunk and we therefore avoid the problem of character boundaries when reading chunks of bytes in UTF-8 encoding:

heuristic_encoding_file(FileName) ->
    {ok,F} = file:open(FileName,[read,binary]),
    loop_through_file(F,<<>>,file:read(F,1024)).

loop_through_file(_,<<>>,eof) ->
    utf8;
loop_through_file(_,_,eof) ->
    latin1;
loop_through_file(F,Acc,{ok,Bin}) when is_binary(Bin) ->
    case unicode:characters_to_binary([Acc,Bin]) of
	{error,_,_} ->
	    latin1;
	{incomplete,_,Rest} ->
	    loop_through_file(F,Rest,file:read(F,1024));
	Res when is_binary(Res) ->
	    loop_through_file(F,<<>>,file:read(F,1024))
    end.

Another option is to try to read the whole file in UTF-8 encoding and see if it fails. Here we need to read the file using function io:get_chars/3, as we have to read characters with a code point > 255:

heuristic_encoding_file2(FileName) ->
    {ok,F} = file:open(FileName,[read,binary,{encoding,utf8}]),
    loop_through_file2(F,io:get_chars(F,'',1024)).

loop_through_file2(_,eof) ->
    utf8;
loop_through_file2(_,{error,_Err}) ->
    latin1;
loop_through_file2(F,Bin) when is_binary(Bin) ->
    loop_through_file2(F,io:get_chars(F,'',1024)).

Lists of UTF-8 Bytes

For various reasons, you can sometimes have a list of UTF-8 bytes. This is not a regular string of Unicode characters, as each list element does not contain one character. Instead you get the "raw" UTF-8 encoding that you have in binaries. This is easily converted to a proper Unicode string by first converting byte per byte into a binary, and then converting the binary of UTF-8 encoded characters back to a Unicode string:

utf8_list_to_string(StrangeList) ->
  unicode:characters_to_list(list_to_binary(StrangeList)).

Double UTF-8 Encoding

When working with binaries, you can get the horrible "double UTF-8 encoding", where strange characters are encoded in your binaries or files. In other words, you can get a UTF-8 encoded binary that for the second time is encoded as UTF-8. A common situation is where you read a file, byte by byte, but the content is already UTF-8. If you then convert the bytes to UTF-8, using, for example, the unicode module, or by writing to a file opened with option {encoding,utf8}, you have each byte in the input file encoded as UTF-8, not each character of the original text (one character can have been encoded in many bytes). There is no real remedy for this other than to be sure of which data is encoded in which format, and never convert UTF-8 data (possibly read byte by byte from a file) into UTF-8 again.

By far the most common situation where this occurs, is when you get lists of UTF-8 instead of proper Unicode strings, and then convert them to UTF-8 in a binary or on a file:

wrong_thing_to_do() ->
  {ok,Bin} = file:read_file("an_utf8_encoded_file.txt"),
  MyList = binary_to_list(Bin), %% Wrong! It is an utf8 binary!
  {ok,C} = file:open("catastrophe.txt",[write,{encoding,utf8}]),
  io:put_chars(C,MyList), %% Expects a Unicode string, but get UTF-8
                          %% bytes in a list!
  file:close(C). %% The file catastrophe.txt contains more or less unreadable
                 %% garbage!

Ensure you know what a binary contains before converting it to a string. If no other option exists, try heuristics:

if_you_can_not_know() ->
  {ok,Bin} = file:read_file("maybe_utf8_encoded_file.txt"),
  MyList = case unicode:characters_to_list(Bin) of
    L when is_list(L) ->
      L;
    _ ->
      binary_to_list(Bin) %% The file was bytewise encoded
  end,
  %% Now we know that the list is a Unicode string, not a list of UTF-8 bytes
  {ok,G} = file:open("greatness.txt",[write,{encoding,utf8}]),
  io:put_chars(G,MyList), %% Expects a Unicode string, which is what it gets!
  file:close(G). %% The file contains valid UTF-8 encoded Unicode characters!

Uniform Resource Identifiers

Basics

At the time of writing this document, in October 2020, there are two major standards concerning Universal Resource Identifiers and Universal Resource Locators:

RFC 3986 - Uniform Resource Identifier (URI): Generic Syntax

WHAT WG URL - Living standard

The former is a classical standard with a proper formal syntax, using the so called Augmented Backus-Naur Form (ABNF) for describing the grammar, while the latter is a living document describing the current pratice, that is, how a majority of Web browsers work with URIs. WHAT WG URL is Web focused and it has no formal grammar but a plain english description of the algorithms that should be followed.

What is the difference between them, if any? They provide an overlapping definition for resource identifiers and they are not compatible. The uri_string module implements RFC 3986 and the term URI will be used throughout this document. A URI is an identifier, a string of characters that identifies a particular resource.

For a more complete problem statement regarding the URIs check the URL Problem Statement and Directions.

What is a URI?

Let's start with what it is not. It is not the text that you type in the address bar in your Web browser. Web browsers do all possible heuristics to convert the input into a valid URI that could be sent over the network.

A URI is an identifier consisting of a sequence of characters matching the syntax rule named URI in RFC 3986.

It is crucial to clarify that a character is a symbol that is displayed on a terminal or written to paper and should not be confused with its internal representation.

A URI more specifically, is a sequence of characters from a subset of the US ASCII character set. The generic URI syntax consists of a hierarchical sequence of components referred to as the scheme, authority, path, query, and fragment. There is a formal description for each of these components in ABNF notation in RFC 3986:

    URI         = scheme ":" hier-part [ "?" query ] [ "#" fragment ]
    hier-part   = "//" authority path-abempty
                   / path-absolute
                   / path-rootless
                   / path-empty
    scheme      = ALPHA *( ALPHA / DIGIT / "+" / "-" / "." )
    authority   = [ userinfo "@" ] host [ ":" port ]
    userinfo    = *( unreserved / pct-encoded / sub-delims / ":" )

    reserved    = gen-delims / sub-delims
    gen-delims  = ":" / "/" / "?" / "#" / "[" / "]" / "@"
    sub-delims  = "!" / "$" / "&" / "'" / "(" / ")"
                / "*" / "+" / "," / ";" / "="

    unreserved  = ALPHA / DIGIT / "-" / "." / "_" / "~"

The uri_string module

As producing and consuming standard URIs can get quite complex, Erlang/OTP provides a module, uri_string, to handle all the most difficult operations such as parsing, recomposing, normalizing and resolving URIs against a base URI.

The API functions in uri_string work on two basic data types uri_string() and uri_map(). uri_string() represents a standard URI, while uri_map() is a wider datatype, that can represent URI components using Unicode characters. uri_map() is a convenient choice for enabling operations such as producing standard compliant URIs out of components that have special or Unicode characters. It is easier to explain this by an example.

Let's say that we would like to create the following URI and send it over the network: http://cities/örebro?foo bar. This is not a valid URI as it contains characters that are not allowed in a URI such as "ö" and the space. We can verify this by parsing the URI:

  1> uri_string:parse("http://cities/örebro?foo bar").
  {error,invalid_uri,":"}

The URI parser tries all possible combinations to interpret the input and fails at the last attempt when it encounters the colon character ":". Note, that the inital fault occurs when the parser attempts to interpret the character "ö" and after a failure back-tracks to the point where it has another possible parsing alternative.

The proper way to solve this problem is to use uri_string:recompose/1 with a uri_map() as input:

  2> uri_string:recompose(#{scheme => "http", host => "cities", path => "/örebro",
  query => "foo bar"}).
  "http://cities/%C3%B6rebro?foo%20bar"

The result is a valid URI where all the special characters are encoded as defined by the standard. Applying uri_string:parse/1 and uri_string:percent_decode/1 on the URI returns the original input:

  3> uri_string:percent_decode(uri_string:parse("http://cities/%C3%B6rebro?foo%20bar")).
  #{host => "cities",path => "/örebro",query => "foo bar",
  scheme => "http"}

This symmetric property is heavily used in our property test suite.

Percent-encoding

As you have seen in the previous chapter, a standard URI can only contain a strict subset of the US ASCII character set, moreover the allowed set of characters is not the same in the different URI components. Percent-encoding is a mechanism to represent a data octet in a component when that octet's corresponding character is outside of the allowed set or is being used as a delimiter. This is what you see when "ö" is encoded as %C3%B6 and space as %20. Most of the API functions are expecting UTF-8 encoding when handling percent-encoded triplets. The UTF-8 encoding of the Unicode character "ö" is two octets: OxC3 0xB6. The character space is in the first 128 characters of Unicode and it is encoded using a single octet 0x20.

Note

Unicode is backward compatible with ASCII, the encoding of the first 128 characters is the same binary value as in ASCII.

It is a major source of confusion exactly which characters will be percent-encoded. In order to make it easier to answer this question the library provides a utility function, uri_string:allowed_characters/0, that lists the allowed set of characters in each major URI component, and also in the most important standard character sets.

    1> uri_string:allowed_characters().
    [{scheme,
     "+-.0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz"},
    {userinfo,
     "!$%&'()*+,-.0123456789:;=ABCDEFGHIJKLMNOPQRSTUVWXYZ_abcdefghijklmnopqrstuvwxyz~"},
    {host,
     "!$&'()*+,-.0123456789:;=ABCDEFGHIJKLMNOPQRSTUVWXYZ_abcdefghijklmnopqrstuvwxyz~"},
    {ipv4,".0123456789"},
    {ipv6,".0123456789:ABCDEFabcdef"},
    {regname,
     "!$%&'()*+,-.0123456789;=ABCDEFGHIJKLMNOPQRSTUVWXYZ_abcdefghijklmnopqrstuvwxyz~"},
    {path,
     "!$%&'()*+,-./0123456789:;=@ABCDEFGHIJKLMNOPQRSTUVWXYZ_abcdefghijklmnopqrstuvwxyz~"},
    {query,
     "!$%&'()*+,-./0123456789:;=?@ABCDEFGHIJKLMNOPQRSTUVWXYZ_abcdefghijklmnopqrstuvwxyz~"},
    {fragment,
     "!$%&'()*+,-./0123456789:;=?@ABCDEFGHIJKLMNOPQRSTUVWXYZ_abcdefghijklmnopqrstuvwxyz~"},
    {reserved,"!#$&'()*+,/:;=?@[]"},
    {unreserved,
     "-.0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ_abcdefghijklmnopqrstuvwxyz~"}]

If a URI component has a character that is not allowed, it will be percent-encoded when the URI is produced:

    2> uri_string:recompose(#{scheme => "https", host => "local#host", path => ""}).
    "https://local%23host"

Consuming a URI containing percent-encoded triplets can take many steps. The following example shows how to handle an input URI that is not normalized and contains multiple percent-encoded triplets. First, the input uri_string() is to be parsed into a uri_map(). The parsing only splits the URI into its components without doing any decoding:

    3> uri_string:parse("http://%6C%6Fcal%23host/%F6re%26bro%20").
    #{host => "%6C%6Fcal%23host",path => "/%F6re%26bro%20",
      scheme => "http"}}

The input is a valid URI but how can you decode those percent-encoded octets? You can try to normalize the input with uri_string:normalize/1. The normalize operation decodes those percent-encoded triplets that correspond to a character in the unreserved set. Normalization is a safe, idempotent operation that converts a URI into its canonical form:

    4> uri_string:normalize("http://%6C%6Fcal%23host/%F6re%26bro%20").
    "http://local%23host/%F6re%26bro%20"
    5> uri_string:normalize("http://%6C%6Fcal%23host/%F6re%26bro%20", [return_map]).
    #{host => "local%23host",path => "/%F6re%26bro%20",
      scheme => "http"}

There are still a few percent-encoded triplets left in the output. At this point, when the URI is already parsed, it is safe to apply application specific decoding on the remaining character triplets. Erlang/OTP provides a function, uri_string:percent_decode/1 for raw percent decoding that you can use on the host and path components, or on the whole map:

    6> uri_string:percent_decode("local%23host").
    "local#host"
    7> uri_string:percent_decode("/%F6re%26bro%20").
    {error,invalid_utf8,<<"/öre&bro ">>}
    8> uri_string:percent_decode(#{host => "local%23host",path => "/%F6re%26bro%20",
    scheme => "http"}).
    {error,{invalid,{path,{invalid_utf8,<<"/öre&bro ">>}}}}

The host was successfully decoded but the path contains at least one character with non-UTF-8 encoding. In order to be able to decode this, you have to make assumptions about the encoding used in these triplets. The most obvious choice is latin-1, so you can try uri_string:transcode/2, to transcode the path to UTF-8 and run the percent-decode operation on the transcoded string:

    9> uri_string:transcode("/%F6re%26bro%20", [{in_encoding, latin1}]).
    "/%C3%B6re%26bro%20"
    10> uri_string:percent_decode("/%C3%B6re%26bro%20").
    "/öre&bro "

It is important to emphasize that it is not safe to apply uri_string:percent_decode/1 directly on an input URI:

    11> uri_string:percent_decode("http://%6C%6Fcal%23host/%C3%B6re%26bro%20").
    "http://local#host/öre&bro "
    12> uri_string:parse("http://local#host/öre&bro ").
    {error,invalid_uri,":"}

Note

Percent-encoding is implemented in uri_string:recompose/1 and it happens when converting a uri_map() into a uri_string(). Applying any percent-encoding directly on an input URI would not be safe just as in the case of uri_string:percent_decode/1, the output could be an invalid URI. Quoting functions allow users to perform raw percent encoding and decoding on application data which cannot be handled automatically by uri_string:recompose/1. For example in scenario when user would need to use '/' or sub-delimeter as data rather than delimeter in a path component.

Normalization

Normalization is the operation of converting the input URI into a canonical form and keeping the reference to the same underlying resource. The most common application of normalization is determining whether two URIs are equivalent without accessing their referenced resources.

Normalization has 6 distinct steps. First the input URI is parsed into an intermediate form that can handle Unicode characters. This datatype is the uri_map(), that can hold the components of the URI in map elements of type unicode:chardata/0. After having the intermediate form, a sequence of normalization algorithms are applied to the individual URI components:

Case normalization - Converts the scheme and host components to lower case as they are not case sensitive.

Percent-encoding normalization - Decodes percent-encoded triplets that correspond to characters in the unreserved set.

Scheme-based normalization - Applying rules for the schemes http, https, ftp, ssh, sftp and tftp.

Path segment normalization - Converts the path into a canonical form.

After these steps, the intermediate data structure, an uri_map(), is fully normalized. The last step is applying uri_string:recompose/1 that converts the intermediate structure into a valid canonical URI string.

Notice the order, the uri_string:normalize(URIMap, [return_map]) that we used many times in this user guide is a shortcut in the normalization process returning the intermediate datastructure, and allowing us to inspect and apply further decoding on the remaining percent-encoded triplets.

    13> uri_string:normalize("hTTp://LocalHost:80/%c3%B6rebro/a/../b").
    "http://localhost/%C3%B6rebro/b"
    14> uri_string:normalize("hTTp://LocalHost:80/%c3%B6rebro/a/../b", [return_map]).
    #{host => "localhost",path => "/%C3%B6rebro/b",
      scheme => "http"}

Special considerations

The current URI implementation provides support for producing and consuming standard URIs. The API is not meant to be directly exposed in a Web browser's address bar where users can basically enter free text. Application designers shall implement proper heuristics to map the input into a parsable URI.

PCRE2 Migration

Starting with Erlang/OTP 28, the underlying library for handling regular expressions via the re module changes from PCRE to PCRE2. This upgrade brings Erlang's regular expression capabilities more in line with modern standards, particularly Perl, but also introduces several breaking changes and differences in behavior compared to PCRE.

A key philosophical difference is that PCRE2 is much stricter about pattern syntax. Invalid constructs that PCRE might have ignored or treated as literal characters will now typically raise compilation errors, similar to using Perl in strict mode.

Below is a summary of notable incompatibilities and behavioral changes:

Stricter Error Handling & Syntax Validation

Invalid Escapes: Undocumented escape sequences (for example, \M, \i) are now treated as errors. PCRE often treated these as literal characters (for example, \M became M).

Invalid Escapes in Character Classes: Using sequences that are not valid within a character class (for example, anchors like \B, sequences like \R or \X) is now an error (for example, [\B]). PCRE might have treated these literally.

Invalid Character Ranges: Ranges where the start point is logically after the end point, or involving incompatible types (for example, [\d-a]), are now errors. PCRE might have interpreted this as matching a digit, a hyphen, or 'a' literally.

Invalid Backreferences: Using a backreference to a non-existent capturing group (for example, \8 when only 7 groups exist) is now treated as an error.

Invalid Control Characters (\cx): The \c escape must be followed by a character that maps to a valid control character (typically ASCII characters @ through _, corresponding to \x00 through \x1F, and ? for \x7F). Using characters in the range 127-255 will result in an error.

Syntax Changes & Requirements

\x Requires Hex Digits: The \x escape now must be followed by hexadecimal digits. Use \xNN (one or two digits) or \x{HHHH} (variable number of digits in braces). Using \x alone is an error.

\N (Match Non-Newline) in Character Classes: The shorthand \N is not allowed directly within a character class (for example, [\N] is invalid). However, the named Unicode sequence \N{U+...} is allowed (for example, [\N{U+0041}] to match 'A').

Empty Group Names: Defining capturing groups with empty names using (?''...) syntax is no longer supported and will cause an error.

Option Handling

Compile-Time vs. Run-Time Options: Options affecting newline conventions ({newline, _}) or backslash R behavior (bsr_anycrlf, bsr_unicode) only control pattern compilation. If a pattern is pre-compiled using re:compile/2, passing incompatible options for these settings later to re:run/3, re:replace/4, or re:split/3 will result in an error. If present, the options must match those used at compile time.

Behavioral Changes & Feature Restrictions

\K in Lookarounds: The \K escape (reset match start) cannot be used inside lookahead (?=...), (?!...) or lookbehind (?<=...), (?<!...) assertions.

\K in global match: does no longer produce a duplicate match.

(*COMMIT) with no_start_optimize: The interaction between the (*COMMIT) verb and the no_start_optimize option may lead to different behavior than might be expected based on previous versions or other regex engines. Careful testing is advised if using this combination.

re:split/3 with Branch Reset Groups ((?|...)): The behavior of re:split/3 when using branch reset groups combined with backreferences within the pattern may differ from PCRE and potentially current Perl versions.

Example: For re:split("abcabc", "(?|(abc)|(xyz))\\1", [{return, list}]):

PCRE2 (OTP 28) returns: ["", "abc", ""]

Some interpretations might expect ["", "abc", "", ""] (empty binary instead of undef for the non-participating capture group within the split context). Reference: Perl Issue #22912

Default Handling of Bytes 128-255 Changed: In OTP 28, when the unicode option is not used, character properties \w, \s, and so on, and option caseless now operate strictly under ASCII rules (bytes 0-127 only). That means that \w will not match latin-1 locale characters such as åäö.

Unicode Property Updates (\p{...}, \P{...})

PCRE2 uses updated Unicode character property data. This means the set of characters matched by properties like \p{L} (Letter), \p{N} (Number), script names (\p{Arabic}), etc., may have changed slightly. Specific examples noted:

\p{Lo} (Letter, Other) now includes characters like U+4DB6.

\p{Arabic} script now includes U+061C (Arabic Letter Mark) and formatting characters U+0650 - U+0655.

\p{Common} script (characters used across multiple scripts) no longer includes U+0589 (Armenian Full Stop). (Verify character if critical).

Erlang-Specific Changes

Compiled Patterns Should Not be Shared: Sharing the result of re:compile/2 across node boundaries or persisting it has never been officially supported, although it might have accidentally worked between certain previous versions. The internal format produced by re:compile/2 has changed, and will not work on older OTP versions or across nodes. Relying on this undocumented behavior must be changed to compile the regular expression on the node instance where it is intended to be used.

assert.hrl

Assert macros.

Description

The include file assert.hrl provides macros for inserting assertions in your program code.

Include the following directive in the module from which the function is called:

-include_lib("stdlib/include/assert.hrl").

When an assertion succeeds, the assert macro yields the atom ok. When an assertion fails, an exception of type error is generated. The associated error term has the form {Macro, Info}. Macro is the macro name, for example, assertEqual. Info is a list of tagged values, such as [{module, M}, {line, L}, ...], which gives more information about the location and cause of the exception. All entries in the Info list are optional; do not rely programmatically on any of them being present.

Each assert macro has a corresponding version with an extra argument, for adding comments to assertions. These can for example be printed as part of error reports, to clarify the meaning of the check that failed. For example, ?assertEqual(0, fib(0), "Fibonacci is defined for zero"). The comment text can be any character data (string, UTF8-binary, or deep list of such data), and will be included in the error term as {comment, Text}.

If the macro NOASSERT is defined when assert.hrl is read by the compiler, the macros are defined as equivalent to the atom ok. The test will not be performed and there is no cost at runtime.

For example, using erlc to compile your modules, the following disables all assertions:

erlc -DNOASSERT=true *.erl

(The value of NOASSERT does not matter, only the fact that it is defined.)

A few other macros also have effect on the enabling or disabling of assertions:

If NODEBUG is defined, it implies NOASSERT (unless DEBUG is also defined, which overrides NODEBUG).

If ASSERT is defined, it overrides NOASSERT, that is, the assertions remain enabled.

If you prefer, you can thus use only DEBUG/NODEBUG as the main flags to control the behavior of the assertions (which is useful if you have other compiler conditionals or debugging macros controlled by those flags), or you can use ASSERT/NOASSERT to control only the assert macros.

Macros

assert(BoolExpr)

assert(BoolExpr, Comment) - Tests that BoolExpr completes normally returning true.

assertNot(BoolExpr)

assertNot(BoolExpr, Comment) - Tests that BoolExpr completes normally returning false.

assertMatch(GuardedPattern, Expr)

assertMatch(GuardedPattern, Expr, Comment) - Tests that Expr completes normally yielding a value that matches GuardedPattern, for example:

?assertMatch({bork, _}, f())

Notice that a guard when ... can be included:

?assertMatch({bork, X} when X > 0, f())

assertNotMatch(GuardedPattern, Expr)

assertNotMatch(GuardedPattern, Expr, Comment) - Tests that Expr completes normally yielding a value that does not match GuardedPattern.

As in assertMatch, GuardedPattern can have a when part.

assertEqual(ExpectedValue, Expr)

assertEqual(ExpectedValue, Expr, Comment) - Tests that Expr completes normally yielding a value that is exactly equal to ExpectedValue.

assertNotEqual(ExpectedValue, Expr)

assertNotEqual(ExpectedValue, Expr, Comment) - Tests that Expr completes normally yielding a value that is not exactly equal to ExpectedValue.

assertException(Class, Term, Expr)

assertException(Class, Term, Expr, Comment) - Tests that Expr completes abnormally with an exception of type Class and with the associated Term. The assertion fails if Expr raises a different exception or if it completes normally returning any value.

Notice that both Class and Term can be guarded patterns, as in assertMatch.

assertNotException(Class, Term, Expr)

assertNotException(Class, Term, Expr, Comment) - Tests that Expr does not evaluate abnormally with an exception of type Class and with the associated Term. The assertion succeeds if Expr raises a different exception or if it completes normally returning any value.

As in assertException, both Class and Term can be guarded patterns.

assertError(Term, Expr)

assertError(Term, Expr, Comment) - Equivalent to assertException(error, Term, Expr)

assertExit(Term, Expr)

assertExit(Term, Expr, Comment) - Equivalent to assertException(exit, Term, Expr)

assertThrow(Term, Expr)

assertThrow(Term, Expr, Comment) - Equivalent to assertException(throw, Term, Expr)

See Also

compile, erlc(3)

erl_tar

Unix 'tar' utility for reading and writing tar archives.

This module archives and extract files to and from a tar file. This module supports reading most common tar formats, namely v7, STAR, USTAR, and PAX, as well as some of GNU tar's extensions to the USTAR format (sparse files most notably). It produces tar archives in USTAR format, unless the files being archived require PAX format due to restrictions in USTAR (such as unicode metadata, filename length, and more). As such, erl_tar supports tar archives produced by most all modern tar utilities, and produces tarballs which should be similarly portable.

By convention, the name of a tar file is to end in ".tar". To abide to the convention, add ".tar" to the name.

Tar files can be created in one operation using function create/2 or create/3.

Alternatively, for more control, use functions open/2, add/3,4, and close/1.

To extract all files from a tar file, use function extract/1. To extract only some files or to be able to specify some more options, use function extract/2.

To return a list of the files in a tar file, use function table/1 or table/2. To print a list of files to the Erlang shell, use function t/1 or tt/1.

To convert an error term returned from one of the functions above to a readable message, use function format_error/1.

Unicode Support

If file:native_name_encoding/0 returns utf8, path names are encoded in UTF-8 when creating tar files, and path names are assumed to be encoded in UTF-8 when extracting tar files.

If file:native_name_encoding/0 returns latin1, no translation of path names is done.

Unicode metadata stored in PAX headers is preserved

Other Storage Media

The ftp module normally accesses the tar file on disk using the file module. When other needs arise, you can define your own low-level Erlang functions to perform the writing and reading on the storage media; use function init/3.

An example of this is the SFTP support in ssh_sftp:open_tar/3. This function opens a tar file on a remote machine using an SFTP channel.

Limitations

If you must remain compatible with the USTAR tar format, you must ensure file paths being stored are less than 255 bytes in total, with a maximum filename component length of 100 bytes. USTAR uses a header field (prefix) in addition to the name field, and splits file paths longer than 100 bytes into two parts. This split is done on a directory boundary, and is done in such a way to make the best use of the space available in those two fields, but in practice this will often mean that you have less than 255 bytes for a path. erl_tar will automatically upgrade the format to PAX to handle longer filenames, so this is only an issue if you need to extract the archive with an older implementation of erl_tar or tar which does not support PAX. In this case, the PAX headers will be extracted as regular files, and you will need to apply them manually.

Like the above, if you must remain USTAR compatible, you must also ensure that paths for symbolic/hard links are no more than 100 bytes, otherwise PAX headers will be used.

Summary

Types

add_opt()

create_opt()

extract_opt()

file_op()

filelist()

gid()

mode()

name_in_archive()

open_type()

tar_descriptor()

tar_entry()

tar_time()

typeflag()

uid()

user_data()

Functions

add(TarDescriptor, Name, Options)

Equivalent to add/4.

add(TarDescriptor, Filename, NameInArchive, Options)

Adds a file to a tar file that has been opened for writing by open/1.

close(TarDescriptor)

Closes a tar file opened by open/2.

create(Name, FileList)

Creates a tar file and archives the files whose names are specified in FileList into it. The files can either be read from disk or be specified as binaries.

create(Name, FileList, Options)

Creates a tar file and archives the files whose names are specified in FileList into it. The files can either be read from disk or be specified as binaries.

extract(Open)

Extracts all files from a tar archive.

extract/2

Extracts files from a tar archive.

format_error/1

Converts an error reason term to a human-readable error message string.

init/3

The Fun is the definition of what to do when the different storage operations functions are to be called from the higher tar handling functions (such as add/3, add/4, and close/1).

open/2

Creates a tar file for writing (any existing file with the same name is truncated).

t(Name)

Prints the names of all files in the tar file Name to the Erlang shell (similar to "tar t").

table(Open)

Equivalent to table(Open, []).

table(Name, Opts)

Retrieves the names of all files in the tar file Name.

tt(Name)

Prints names and information about all files in the tar file Name to the Erlang shell (similar to "tar tv").

Types

add_opt()

(not exported)

-type add_opt() ::
          dereference | verbose |
          {chunks, pos_integer()} |
          {atime, non_neg_integer()} |
          {mtime, non_neg_integer()} |
          {ctime, non_neg_integer()} |
          {uid, non_neg_integer()} |
          {gid, non_neg_integer()}.

create_opt()

(not exported)

-type create_opt() :: compressed | cooked | dereference | verbose.

extract_opt()

(not exported)

-type extract_opt() ::
          {cwd, string()} |
          {files, [name_in_archive()]} |
          compressed | cooked | memory | keep_old_files | verbose.

file_op()

(not exported)

-type file_op() ::
          fun((write | close | read2 | position,
               {user_data(), iodata()} |
               user_data() |
               {user_data(), non_neg_integer()} |
               {user_data(), non_neg_integer()}) ->
                  ok | eof | {ok, string() | binary()} | {ok, non_neg_integer()} | {error, term()}).

filelist()

(not exported)

-type filelist() :: [file:filename() | {name_in_archive(), file:filename_all()}].

gid()

(not exported)

-type gid() :: non_neg_integer().

mode()

(not exported)

-type mode() :: non_neg_integer().

name_in_archive()

(not exported)

-type name_in_archive() :: string().

open_type()

(not exported)

-type open_type() :: file:filename_all() | {binary, binary()} | {file, file:io_device()}.

tar_descriptor()

-opaque tar_descriptor()

tar_entry()

(not exported)

-type tar_entry() ::
          {Name :: name_in_archive(),
           Type :: typeflag(),
           Size :: non_neg_integer(),
           MTime :: tar_time(),
           Mode :: mode(),
           Uid :: uid(),
           Gid :: gid()}.

tar_time()

(not exported)

-type tar_time() :: non_neg_integer().

typeflag()

(not exported)

-type typeflag() :: regular | link | symlink | char | block | directory | fifo | reserved | unknown.

uid()

(not exported)

-type uid() :: non_neg_integer().

user_data()

(not exported)

-type user_data() :: term().

Functions

add(TarDescriptor, Name, Options)

-spec add(TarDescriptor, Name, Options) -> ok | {error, term()}
             when
                 TarDescriptor :: tar_descriptor(),
                 Name :: name_in_archive() | {name_in_archive(), file:filename_all()},
                 Options :: [add_opt()].

Equivalent to add/4.

If Name is name_in_archive/0, then add(TarDescriptor, Name, Name, Options) is called.

If Name is a two tuple then add(TarDescriptor, NameInArchive, Name, Options) is called.

add(TarDescriptor, Filename, NameInArchive, Options)

-spec add(TarDescriptor, Filename, NameInArchive, Options) -> ok | {error, term()}
             when
                 TarDescriptor :: tar_descriptor(),
                 Filename :: file:filename_all(),
                 NameInArchive :: name_in_archive(),
                 Options :: [add_opt()].

Adds a file to a tar file that has been opened for writing by open/1.

NameInArchive is the name under which the file becomes stored in the tar file. The file gets this name when it is extracted from the tar file.

Options:

dereference - By default, symbolic links are stored as symbolic links in the tar file. To override the default and store the file that the symbolic link points to into the tar file, use option dereference.

verbose - Prints an informational message about the added file.

{chunks,ChunkSize} - Reads data in parts from the file. This is intended for memory-limited machines that, for example, builds a tar file on a remote machine over SFTP, see ssh_sftp:open_tar/3.

{atime,non_neg_integer()} - Sets the last time, as POSIX time, when the file was read. See also file:read_file_info/1.

{mtime,non_neg_integer()} - Sets the last time, as POSIX time, when the file was written. See also file:read_file_info/1.

{ctime,non_neg_integer()} - Sets the time, as POSIX time, when the file was created. See also file:read_file_info/1.

{uid,non_neg_integer()} - Sets the file owner. file:read_file_info/1.

{gid,non_neg_integer()} - Sets the group that the file owner belongs to. file:read_file_info/1.

close(TarDescriptor)

-spec close(TarDescriptor :: tar_descriptor()) -> ok | {error, term()}.

Closes a tar file opened by open/2.

create(Name, FileList)

-spec create(file:filename_all(), filelist()) -> ok | {error, {string(), term()}}.

Creates a tar file and archives the files whose names are specified in FileList into it. The files can either be read from disk or be specified as binaries.

create(Name, FileList, Options)

-spec create(file:filename_all(), filelist(), [create_opt()]) ->
                ok | {error, term()} | {error, {string(), term()}}.

Creates a tar file and archives the files whose names are specified in FileList into it. The files can either be read from disk or be specified as binaries.

The options in OptionList modify the defaults as follows:

compressed - The entire tar file is compressed, as if it has been run through the gzip program. To abide to the convention that a compressed tar file is to end in ".tar.gz" or ".tgz", add the appropriate extension.

cooked - By default, function open/2 opens the tar file in raw mode, which is faster but does not allow a remote (Erlang) file server to be used. Adding cooked to the mode list overrides the default and opens the tar file without option raw.

dereference - By default, symbolic links are stored as symbolic links in the tar file. To override the default and store the file that the symbolic link points to into the tar file, use option dereference.

verbose - Prints an informational message about each added file.

extract(Open)

-spec extract(Open :: open_type()) -> ok | {error, term()}.

Extracts all files from a tar archive.

If argument Name is specified as {binary,Binary}, the contents of the binary is assumed to be a tar archive.

If argument Name is specified as {file,Fd}, Fd is assumed to be a file descriptor returned from function file:open/2.

Otherwise, Name is to be a filename.

Note

Leading slashes in tar member names will be removed before writing the file. That is, absolute paths will be turned into relative paths. There will be an info message written to the error logger when paths are changed in this way.

Warning

The compressed and cooked flags are invalid when passing a file descriptor with {file,Fd}. The file is assumed to have been opened with the appropriate flags.

extract/2

-spec extract(Open :: open_type(), [extract_opt()]) ->
                 {ok, [{string(), binary()}]} | {error, term()} | ok.

Extracts files from a tar archive.

If argument Name is specified as {binary,Binary}, the contents of the binary is assumed to be a tar archive.

If argument Name is specified as {file,Fd}, Fd is assumed to be a file descriptor returned from function file:open/2.

Otherwise, Name is to be a filename.

The following options modify the defaults for the extraction as follows:

{cwd,Cwd} - Files with relative filenames are by default extracted to the current working directory. With this option, files are instead extracted into directory Cwd.

{files,FileList} - By default, all files are extracted from the tar file. With this option, only those files are extracted whose names are included in FileList.

compressed - With this option, the file is uncompressed while extracting. If the tar file is not compressed, this option is ignored.

cooked - By default, function open/2 function opens the tar file in raw mode, which is faster but does not allow a remote (Erlang) file server to be used. Adding cooked to the mode list overrides the default and opens the tar file without option raw.

memory - Instead of extracting to a directory, this option gives the result as a list of tuples {Filename, Binary}, where Binary is a binary containing the extracted data of the file named Filename in the tar file.

keep_old_files - By default, all existing files with the same name as files in the tar file are overwritten. With this option, existing files are not overwriten.

verbose - Prints an informational message for each extracted file.

Warning

The compressed and cooked flags are invalid when passing a file descriptor with {file,Fd}. The file is assumed to have been opened with the appropriate flags.

format_error/1

-spec format_error(term()) -> string().

Converts an error reason term to a human-readable error message string.

init/3

(since OTP 17.4)

-spec init(UserData :: user_data(), write | read, file_op()) -> {ok, tar_descriptor()} | {error, badarg}.

The Fun is the definition of what to do when the different storage operations functions are to be called from the higher tar handling functions (such as add/3, add/4, and close/1).

The Fun is called when the tar function wants to do a low-level operation, like writing a block to a file. The Fun is called as Fun(Op, {UserData,Parameters...}), where Op is the operation name, UserData is the term passed as the first argument to init/1 and Parameters... are the data added by the tar function to be passed down to the storage handling function.

Parameter UserData is typically the result of opening a low-level structure like a file descriptor or an SFTP channel id. The different Fun clauses operate on that very term.

The following are the fun clauses parameter lists:

(write, {UserData,DataToWrite}) - Writes term DataToWrite using UserData.

(close, UserData) - Closes the access.

(read2, {UserData,Size}) - Reads using UserData but only Size bytes. Notice that there is only an arity-2 read function, not an arity-1 function.

(position,{UserData,Position}) - Sets the position of UserData as defined for files in file:position/2

Example:

The following is a complete Fun parameter for reading and writing on files using the file module:

ExampleFun =
   fun(write, {Fd,Data}) ->  file:write(Fd, Data);
      (position, {Fd,Pos}) -> file:position(Fd, Pos);
      (read2, {Fd,Size}) -> file:read(Fd, Size);
      (close, Fd) -> file:close(Fd)
   end

Here Fd was specified to function init/3 as:

{ok,Fd} = file:open(Name, ...).
{ok,TarDesc} = erl_tar:init(Fd, write, ExampleFun),

TarDesc is then used:

erl_tar:add(TarDesc, SomeValueIwantToAdd, FileNameInTarFile),
...,
erl_tar:close(TarDesc)

When the erl_tar core wants to, for example, write a piece of Data, it would call ExampleFun(write, {UserData,Data}).

Note

This example with the file module operations is not necessary to use directly, as that is what function open/2 in principle does.

Warning

The TarDescriptor term is not a file descriptor. You are advised not to rely on the specific contents of this term, as it can change in future Erlang/OTP releases when more features are added to this module.

open/2

-spec open(Open :: open_type(), [write | compressed | cooked]) ->
              {ok, tar_descriptor()} | {error, term()}.

Creates a tar file for writing (any existing file with the same name is truncated).

By convention, the name of a tar file is to end in ".tar". To abide to the convention, add ".tar" to the name.

Except for the write atom, the following atoms can be added to OpenModeList:

compressed - The entire tar file is compressed, as if it has been run through the gzip program. To abide to the convention that a compressed tar file is to end in ".tar.gz" or ".tgz", add the appropriate extension.

cooked - By default, the tar file is opened in raw mode, which is faster but does not allow a remote (Erlang) file server to be used. Adding cooked to the mode list overrides the default and opens the tar file without option raw.

To add one file at the time into an opened tar file, use function add/3,4. When you are finished adding files, use function close/1 to close the tar file.

Warning

The compressed and cooked flags are invalid when passing a file descriptor with {file,Fd}. The file must already be opened with the appropriate flags.

Warning

The TarDescriptor term is not a file descriptor. You are advised not to rely on the specific contents of this term, as it can change in future Erlang/OTP releases when more features are added to this module.

t(Name)

-spec t(file:filename()) -> ok | {error, term()}.

Prints the names of all files in the tar file Name to the Erlang shell (similar to "tar t").

table(Open)

-spec table(Open :: open_type()) -> {ok, [name_in_archive()]} | {error, term()}.

Equivalent to table(Open, []).

table(Name, Opts)

-spec table(Open :: open_type(), [compressed | verbose | cooked]) ->
               {ok, [name_in_archive() | tar_entry()]} | {error, term()}.

Retrieves the names of all files in the tar file Name.

tt(Name)

-spec tt(open_type()) -> ok | {error, term()}.

Prints names and information about all files in the tar file Name to the Erlang shell (similar to "tar tv").

rand

Pseudo random number generation

This module provides Pseudo Random Number Generation and implements a number of base generator algorithms. Most are provided through a plug-in framework that adds essential features to the base generators.

PRNGs in general, and so the algorithms in this module, are mostly used for test and simulation. They are designed for good statistical quality and high generation speed.

A generator algorithm, for each iteration, takes a state as input and produces a raw pseudo random number and a new state to be used for the next iteration.

A particular state always produces the same number and new state. The initial state is produced from a seed. This makes it possible to repeat for example a simulation with the same random number sequence, by re-using the same seed. There are also the functions export_seed/0 and export_seed_s/1 that capture the PRNG state in an export_state/0, that can be used to start from a known state.

This property, and others, make the algorithms in this module unsuitable for cryptographical applications, but in the crypto module there are suitable generators, for this module's plug-in framework. See crypto:rand_seed_s/0 and crypto:rand_seed_alg_s/1.

At the end of this module documentation there are some niche algorithms that do not use this module's normal plug-in framework. They are useful for special purposes like fast generation when quality is not essential, for seeding other generators, and such.

Plug-in framework

The raw pseudo random numbers produced by the base generators are only appropriate in some cases such as power of two ranges less than the generator size, and some have quirks, for example weak low bits. Therefore, the Plug-in Framework implements a common API for all base generators, that add essential or useful funcionality:

Keeping the generator state in the process dictionary.

Automatic seeding.

Seeding support for manual seeding to avoid common pitfalls.

Generating integers with uniform distribution, in any range, without bias. The range is not limited; it may be larger than the base generator's size (but that costs some performance).

Generating floating-point numbers with uniform distribution.

Generating floating-point numbers with normal distribution, standard normal distribution or specified mean and variance.

Generating any number of bytes.

Jumping the generator ahead, in algorithms that support that.

Usage and examples

A generator has to be initialized. This is done by one of the seed/1 or seed_s/1 functions, which also select which algorithm to use. The seed/1 functions store the generator and state in the process dictionary, while the seed_s/1 functions only return the state, which requires the calling code to handle the state and updates to it.

The seed functions that do not have a Seed value as an argument create an automatic seed that should be unique to the created generator instance; see seed_s/1.

If an automatic seed is not desired, the seed functions that have a Seed argument can be used. The argument has 3 possible formats; see the seed/0 type description.

Plug-in framework API functions named with the suffix _s take an explicit state as the last argument and return the new state as the last element in the returned tuple. The process dictionary is not used.

Sibling functions without that suffix take an implicit state from and store the new state in the process dictionary, and only return their "interesting " output value. If the process dictionary does not contain a state, seed(default) is implicitly called to create an automatic seed for the default algorithm as initial state.

Usage

First initialize a generator by calling one of the seed functions, which also selects a PRNG algorithm.

Then call a Plug-in framework API function either with an explicit state from the seed function and use the returned new state in the next call, or call an API function without an explicit state argument to operate on the state in the process dictionary.

Examples

%% Generate two uniformly distibuted floating point numbers.
%%
%% By not calling a [seed](`seed/1`) function, this uses
%% the generator state and algorithm in the process dictionary.
%% If there is no state there, [`seed(default)`](`seed/1`)
%% is implicitly called first:
%%
1> R0 = rand:uniform(),
   is_float(R0) andalso 0.0 =< R0 andalso R0 < 1.0.
true
2> R1 = rand:uniform(),
   is_float(R1) andalso 0.0 =< R1 andalso R1 < 1.0.
true

%% Generate a uniformly distributed integer in the range 1..4711:
%%
3> K0 = rand:uniform(4711),
   is_integer(K0) andalso 1 =< K0 andalso K0 =< 4711.
true

%% Generate a binary with 16 bytes, uniformly distributed:
%%
4> B0 = rand:bytes(16),
   byte_size(B0) == 16.
true

%% Select and initialize a specified algorithm,
%% with an automatic default seed, then generate
%% a floating point number:
%%
5> rand:seed(exro928ss).
6> R2 = rand:uniform(),
   is_float(R2) andalso 0.0 =< R2 andalso R2 < 1.0.
true

%% Select and initialize a specified algorithm
%% with a specified seed, then generate
%% a floating point number:
%%
7> rand:seed(exro928ss, 123456789).
8> R3 = rand:uniform().
0.48303622772415256

%% Select and initialize a specific algorithm,
%% with an automatic default seed, using the functional API
%% with explicit generator state, then generate
%% two floating point numbers.
%%
9>  S0 = rand:seed_s(exsss).
10> {R4, S1} = rand:uniform_s(S0),
    is_float(R4) andalso 0.0 =< R4 andalso R4 < 1.0.
true
11> {R5, S2} = rand:uniform_s(S1),
    is_float(R5) andalso 0.0 =< R5 andalso R5 < 1.0.
true
%% Repeat the first after seed
12> {R4, _} = rand:uniform_s(S0).

%% Generate a standard normal distribution number
%% using the built-in fast Ziggurat Method:
%%
13> {SND0, S3} = rand:normal_s(S2),
    is_float(SND0).
true

%% Generate a normal distribution number
%% with mean -3 and variance 0.5:
%%
14> {ND0, S4} = rand:normal_s(-3, 0.5, S3),
    is_float(ND0).
true

%% Generate a textbook basic form Box-Muller
%% standard normal distribution number, which has the same
%% distribution as the built-in Ziggurat method above,
%% but is much slower:
%%
15> R6 = rand:uniform_real(),
    is_float(R6) andalso 0.0 < R6 andalso R6 < 1.0.
true
16> R7 = rand:uniform(),
    is_float(R7) andalso 0.0 =< R7 andalso R7 < 1.0.
true
%% R6 cannot be equal to 0.0 so math:log/1 will never fail
17> SND1 = math:sqrt(-2 * math:log(R6)) * math:cos(math:pi() * R7).

Algorithms

The base generator algorithms implement the Xoroshiro and Xorshift algorithms by Sebastiano Vigna. During an iteration they generate an integer (at least 58-bit) and operate on a state of several integers. The size of these integers is chosen to not require bignum arithmetic on 64-bit platforms, which facilitates fast integer operations, in particular when handled by the JIT VM.

For most algorithms, jump functions are provided for generating non-overlapping sequences. A jump function perform a calculation equivalent to a large number of repeated state iterations, but execute in a time roughly equivalent to one regular iteration per generator bit.

By using a jump function instead of starting several generators from different seeds it is assured that the generated sequences do not overlap. The alternative of using different seeds may accidentally start the generators in sequence positions that are close to each other, but a jump function jumps to a sequence position very far ahead.

To create numbers with normal distribution the Ziggurat Method by Marsaglia and Tsang is used on the output from a base generator.

The following algorithms are provided:

exsss, the default algorithm (Since OTP 22.0)
Xorshift116**, 58 bits precision and period of 2^116-1.

Jump function: equivalent to 2^64 calls.

This is the Xorshift116 generator combined with the StarStar scrambler from the 2018 paper by David Blackman and Sebastiano Vigna: Scrambled Linear Pseudorandom Number Generators

The generator does not use 58-bit rotates so it is faster than the Xoroshiro116 generator, and when combined with the StarStar scrambler it does not have any weak low bits like exrop (Xoroshiro116+).

Alas, this combination is about 10% slower than exrop, but despite that it is the default algorithm thanks to its statistical qualities.

exro928ss (Since OTP 22.0)
Xoroshiro928**, 58 bits precision and a period of 2^928-1.

Jump function: equivalent to 2^512 calls.

This is a 58 bit version of Xoroshiro1024**, from the 2018 paper by David Blackman and Sebastiano Vigna: Scrambled Linear Pseudorandom Number Generators that on a 64 bit Erlang system executes only about 40% slower than the default exsss algorithm but with much longer period and better statistical properties, but on the flip side a larger state.

Many thanks to Sebastiano Vigna for his help with the 58 bit adaption.

exrop (Since OTP 20.0)
Xoroshiro116+, 58 bits precision and period of 2^116-1.

Jump function: equivalent to 2^64 calls.

exs1024s (Since OTP 20.0)
Xorshift1024*, 64 bits precision and a period of 2^1024-1

Jump function: equivalent to 2^512 calls.

Since this generator operates on 64-bit integers that are bignums on 64 bit platforms, it is much slower than exro928ss above.

exsp (Since OTP 20.0)
Xorshift116+, 58 bits precision and period of 2^116-1

Jump function: equivalent to 2^64 calls.

This is a corrected version of a previous default algorithm (exsplus, deprecated), that was superseded by Xoroshiro116+ (exrop). Since this algorithm does not use rotate operations it executes a little (say < 15%) faster than exrop (that has to do a 58 bit rotate, for which there is no native instruction). See the algorithms' homepage.

Default Algorithm

The current default algorithm is exsss (Xorshift116**). If a specific algorithm is required, ensure to always use seed/1 to initialize the state.

Which algorithm that is the default may change between Erlang/OTP releases, and is selected to be one with high speed, small state and "good enough" statistical properties. So to ensure that the same sequence is reproduced on a later Erlang/OTP release, use a seed/2 or seed_s/2 to select both a specific algorithm and the seed value.

Old Algorithms

Undocumented (old) algorithms are deprecated but still implemented so old code relying on them will produce the same pseudo random sequences as before.

Note

There were a number of problems in the implementation of the now undocumented algorithms, which is why they are deprecated. The new algorithms are a bit slower but do not have these problems:

Uniform integer ranges had a skew in the probability distribution that was not noticable for small ranges but for large ranges less than the generator's precision the probability to produce a low number could be twice the probability for a high.

Uniform integer ranges larger than or equal to the generator's precision used a floating point fallback that only calculated with 52 bits which is smaller than the requested range and therefore all numbers in the requested range were not even possible to produce.

Uniform floats had a non-uniform density so small values for example less than 0.5 had got smaller intervals decreasing as the generated value approached 0.0 although still uniformly distributed for sufficiently large subranges. The new algorithms produces uniformly distributed floats of the form N * 2.0^(-53) hence they are equally spaced.

Quality of the Generated Numbers

Note

The builtin random number generator algorithms are not cryptographically strong. If a cryptographically strong random number generator is needed, use for example crypto:rand_seed_s/0 or crypto:rand_seed_alg_s/1.

For all these generators except exro928ss and exsss the lowest bit(s) have got a slightly less random behaviour than all other bits. 1 bit for exrop (and exsp), and 3 bits for exs1024s. See for example this explanation in the Xoroshiro128+ generator source code:

Beside passing BigCrush, this generator passes the PractRand test suite up to (and included) 16TB, with the exception of binary rank tests, which fail due to the lowest bit being an LFSR; all other bits pass all tests. We suggest to use a sign test to extract a random Boolean value.

If this is a problem; to generate a boolean with these algorithms, use something like this:

(rand:uniform(256) > 128) % -> boolean()

((rand:uniform(256) - 1) bsr 7) % -> 0 | 1

For a general range, with N = 1 for exrop, and N = 3 for exs1024s:

(((rand:uniform(Range bsl N) - 1) bsr N) + 1)

The floating point generating functions in this module waste the lowest bits when converting from an integer so they avoid this snag.

Niche algorithms

The niche algorithms API contains special purpose algorithms that do not use the plug-in framework, mainly for performance reasons.

Since these algorithms lack the plug-in framework support, generating numbers in a range other than the base generator's range may become a problem.

There are at least four ways to do this, assuming the Range is less than the generator's range:

Modulo
To generate a number V in the range 0..Range-1:

Generate a number X.
Use V = X rem Range as your value.

This method uses rem, that is, the remainder of an integer division, which is a slow operation.

Low bits from the generator propagate straight through to the generated value, so if the generator has got weaknesses in the low bits this method propagates them too.

If Range is not a divisor of the generator range, the generated numbers have a bias. Example:

Say the generator generates a byte, that is, the generator range is 0..255, and the desired range is 0..99 (Range = 100). Then there are 3 generator outputs that produce the value 0, these are; 0, 100 and 200. But there are only 2 generator outputs that produce the value 99, which are; 99 and 199. So the probability for a value V in 0..55 is 3/2 times the probability for the other values 56..99.

If Range is much smaller than the generator range, then this bias gets hard to detect. The rule of thumb is that if Range is smaller than the square root of the generator range, the bias is small enough. Example:

A byte generator when Range = 20. There are 12 (256 div 20) possibilities to generate the highest numbers and one more to generate a number V < 16 (256 rem 20). So the probability is 13/12 for a low number versus a high. To detect that difference with some confidence you would need to generate a lot more numbers than the generator range, 256 in this small example.

Truncated multiplication
To generate a number V in the range 0..Range-1, when you have a generator with a power of 2 range (0..2^Bits-1):

Generate a number X.
Use V = X * Range bsr Bits as your value.

If the multiplication X * Range creates a bignum this method becomes very slow.

High bits from the generator propagate through to the generated value, so if the generator has got weaknesses in the high bits this method propagates them too.

If Range is not a divisor of the generator range, the generated numbers have a bias, pretty much as for the Modulo method above.

Shift or mask
To generate a number in a power of 2 range (0..2^RBits-1), when you have a generator with a power of 2 range (0..2^Bits):

Generate a number X.
Use V = X band ((1 bsl RBits)-1) or V = X bsr (Bits-RBits) as your value.

Masking with band preserves the low bits, and right shifting with bsr preserves the high, so if the generator has got weaknesses in high or low bits; choose the right operator.

If the generator has got a range that is not a power of 2 and this method is used anyway, it introduces bias in the same way as for the Modulo method above.

Rejection

Generate a number X.
If X is in the range, use it as your value, otherwise reject it and repeat.

In theory it is not certain that this method will ever complete, but in practice you ensure that the probability of rejection is low. Then the probability for yet another iteration decreases exponentially so the expected mean number of iterations will often be between 1 and 2. Also, since the base generator is a full length generator, a value that will break the loop must eventually be generated.

These methods can be combined, such as using the Modulo method and only if the generator value would create bias use Rejection. Or using Shift or mask to reduce the size of a generator value so that Truncated multiplication will not create a bignum.

The recommended way to generate a floating point number (IEEE 745 Double, that has got a 53-bit mantissa) in the range 0..1, that is 0.0 =< V < 1.0 is to generate a 53-bit number X and then use V = X * (1.0/((1 bsl 53))) as your value. This will create a value of the form N*2^-53 with equal probability for every possible N for the range.

Summary

Types

alg()

alg_handler()

alg_state()

builtin_alg()

dummy_state()

Algorithm specific internal state

export_state()

Algorithm-dependent state that can be printed or saved to file.

exro928_state()

Algorithm specific internal state

exrop_state()

Algorithm specific internal state

exs64_state()

Algorithm specific internal state

exs1024_state()

Algorithm specific internal state

exsplus_state()

Algorithm specific internal state

mwc59_state()

1 .. (16#1ffb072 bsl 29) - 2

seed()

Generator seed value.

splitmix64_state()

Algorithm specific state

state()

Algorithm-dependent state.

uint58()

0 .. (2^58 - 1)

uint64()

0 .. (2^64 - 1)

Niche algorithms API

exsp_jump(AlgState)

Jump the generator state forward.

exsp_next(AlgState)

Generate an Xorshift116+ random integer and new algorithm state.

mwc59(CX0)

Generate a new MWC59 state.

mwc59_float(CX)

Calculate a scrambled float/0 from a MWC59 state.

mwc59_seed()

Create a MWC59 generator state.

mwc59_seed(S)

Create a MWC59 generator state.

mwc59_value32(CX)

Calculate a 32-bit scrambled value from a MWC59 state.

mwc59_value(CX)

Calculate a 59-bit scrambled value from a MWC59 state.

splitmix64_next(AlgState)

Generate a SplitMix64 random 64-bit integer and new algorithm state.

Plug-in framework API

bytes(N)

Generate random bytes as a t:binary(), using the state in the process dictionary.

bytes_s(N, State)

Generate random bytes as a t:binary().

export_seed()

Export the seed value.

export_seed_s(State)

Export the seed value.

jump()

Jump the generator state forward.

jump(State)

Jump the generator state forward.

normal()

Generate a random number with standard normal distribution.

normal(Mean, Variance)

Generate a random number with specified normal distribution 𝒩 (μ, σ²).

normal_s(State)

Generate a random number with standard normal distribution.

normal_s(Mean, Variance, State)

Generate a random number with specified normal distribution 𝒩 (μ, σ²).

seed(Alg_or_State)

Seed the random number generator and select algorithm.

seed(Alg, Seed)

Seed the random number generator and select algorithm.

seed_s(Alg_or_State)

Seed the random number generator and select algorithm.

seed_s(Alg, Seed)

Seed the random number generator and select algorithm.

shuffle(List)

Shuffle a list.

shuffle_s(List, State)

Shuffle a list.

uniform()

Generate a uniformly distributed random number 0.0 =< X < 1.0, using the state in the process dictionary.

uniform(N)

Generate a uniformly distributed random integer 1 =< X =< N, using the state in the process dictionary.

uniform_real()

Generate a uniformly distributed random number 0.0 < X < 1.0, using the state in the process dictionary.

uniform_real_s(State)

Generate a uniformly distributed random number 0.0 < X < 1.0.

uniform_s(State)

Generate a uniformly distributed random number 0.0 =< X < 1.0.

uniform_s(N, State)

Generate a uniformly distributed random integer 1 =< X =< N.

Types

alg()

(since OTP 18.0)

-type alg() :: builtin_alg() | atom().

alg_handler()

(since OTP 18.0)

-type alg_handler() ::
          #{type := alg(),
            bits => non_neg_integer(),
            weak_low_bits => non_neg_integer(),
            max => non_neg_integer(),
            next := fun((alg_state()) -> {non_neg_integer(), alg_state()}),
            uniform => fun((state()) -> {float(), state()}),
            uniform_n => fun((pos_integer(), state()) -> {pos_integer(), state()}),
            jump => fun((state()) -> state())}.

alg_state()

(since OTP 18.0)

-type alg_state() ::
          exsplus_state() |
          exro928_state() |
          exrop_state() |
          exs1024_state() |
          exs64_state() |
          dummy_state() |
          term().

builtin_alg()

(since OTP 18.0)

-type builtin_alg() :: exsss | exro928ss | exrop | exs1024s | exsp | exs64 | exsplus | exs1024 | dummy.

dummy_state()

(since OTP 18.0)

-type dummy_state() :: uint58().

Algorithm specific internal state

export_state()

(since OTP 18.0)

-type export_state() :: {alg(), alg_state()}.

Algorithm-dependent state that can be printed or saved to file.

exro928_state()

(since OTP 18.0)

-opaque exro928_state()

Algorithm specific internal state

exrop_state()

(since OTP 18.0)

-opaque exrop_state()

Algorithm specific internal state

exs64_state()

(since OTP 18.0)

-opaque exs64_state()

Algorithm specific internal state

exs1024_state()

(since OTP 18.0)

-opaque exs1024_state()

Algorithm specific internal state

exsplus_state()

(since OTP 18.0)

-opaque exsplus_state()

Algorithm specific internal state

mwc59_state()

(since OTP 18.0)

-type mwc59_state() :: 1..133850370 bsl 32 - 1 - 1.

1 .. (16#1ffb072 bsl 29) - 2

seed()

(since OTP 18.0)

-type seed() :: [integer()] | integer() | {integer(), integer(), integer()}.

Generator seed value.

A single integer is the easiest to use. It is set as the initial state of a SplitMix64 generator. The sequential output values of that generator are then used for setting the actual generator's internal state, after masking to the proper word size and avoiding zero values, if necessary.

A list of integers sets the generator's internal state directly, after algorithm-dependent checks of the value and masking to the proper word size. The number of integers must be equal to the number of state words in the generator. This format would only be needed in special cases.

A traditional 3-tuple of integers is passed through algorithm-dependent hashing functions to create the generator's initial state. This format is inherited from this module's predecessor, the random module, where the 3-tuple from erlang:now/0 (also now deprectated) was often used for seeding to get some uniqueness.

splitmix64_state()

(since OTP 18.0)

-type splitmix64_state() :: uint64().

Algorithm specific state

state()

(since OTP 18.0)

-type state() :: {alg_handler(), alg_state()}.

Algorithm-dependent state.

uint58()

(since OTP 18.0)

-type uint58() :: 0..1 bsl 58 - 1.

0 .. (2^58 - 1)

uint64()

(since OTP 18.0)

-type uint64() :: 0..1 bsl 64 - 1.

0 .. (2^64 - 1)

Niche algorithms API

exsp_jump(AlgState)

(since OTP 25.0)

-spec exsp_jump(AlgState :: exsplus_state()) -> NewAlgState :: exsplus_state().

Jump the generator state forward.

Performs a State jump calculation that is equivalent to a 2^64 state iterations.

Returns the NewState.

This feature can be used to create many non-overlapping random number sequences from one start state.

See the description of jump functions at the top of this module description.

See exsp_next/1 about why this internal implementation function has been exposed.

exsp_next(AlgState)

(since OTP 25.0)

-spec exsp_next(AlgState :: exsplus_state()) -> {X :: uint58(), NewAlgState :: exsplus_state()}.

Generate an Xorshift116+ random integer and new algorithm state.

From the specified AlgState, generates a random 58-bit integer X and a new algorithm state NewAlgState, according to the Xorshift116+ algorithm.

This is an API function exposing the internal implementation of the exsp algorithm that enables using it without the overhead of the plug-in framework, which might be useful for time critial applications. On a typical 64 bit Erlang VM this approach executes in just above 30% (1/3) of the time for the default algorithm through this module's normal plug-in framework.

To seed this generator use {_, AlgState} = rand:seed_s(exsp) or {_, AlgState} = rand:seed_s(exsp, Seed) with a specific Seed.

Note

This function offers no help in generating a number on a selected range, nor in generating floating point numbers. It is easy to accidentally mess up the statistical properties of this generator or to loose the performance advantage when doing either. See the recipes at the start of this Niche algorithms API description.

Note also the caveat about weak low bits that this generator suffers from.

The generator is exported in this form primarily for performance reasons.

mwc59(CX0)

(since OTP 25.0)

-spec mwc59(CX0 :: mwc59_state()) -> CX1 :: mwc59_state().

Generate a new MWC59 state.

From the specified generator state CX0 generate a new state CX1, according to a Multiply With Carry generator, which is an efficient implementation of a Multiplicative Congruential Generator with a power of 2 multiplier and a prime modulus.

This generator uses the multiplier 2^32 and the modulus 16#7fa6502 * 2^32 - 1, which have been selected, in collaboration with Sebastiano Vigna, to avoid bignum operations and still get good statistical quality. It has been named "MWC59" and can be written as:

C = CX0 bsr 32
X = CX0 band ((1 bsl 32)-1))
CX1 = 16#7fa6502 * X + C

Because the generator uses a multiplier that is a power of 2 it gets statistical flaws for collision tests and birthday spacings tests in 2 and 3 dimensions, and these caveats apply even when looking only at the MWC "digit", that is the low 32 bits (the multiplier) of the generator state. The higher bits of the state are worse.

The quality of the output value improves much by using a scrambler, instead of just taking the low bits. Function mwc59_value32 is a fast scrambler that returns a decent 32-bit number. The slightly slower mwc59_value scrambler returns 59 bits of very good quality, and mwc59_float returns a float/0 of very good quality.

The low bits of the base generator are surprisingly good, so the lowest 16 bits actually pass fairly strict PRNG tests, despite the generator's weaknesses that lie in the high bits of the 32-bit MWC "digit". It is recommended to use rem on the the generator state, or bit mask extracting the lowest bits to produce numbers in a range 16 bits or less. See the recipes at the start of this Niche algorithms API description.

On a typical 64 bit Erlang VM this generator executes in below 8% (1/13) of the time for the default algorithm in the plug-in framework API of this module. With the mwc59_value32 scrambler the total time becomes 16% (1/6), and with mwc59_value it becomes 20% (1/5) of the time for the default algorithm. With mwc59_float the total time is 60% of the time for the default algorithm generating a float/0.

Note

This generator is a niche generator for high speed applications. It has a much shorter period than the default generator, which in itself is a quality concern, although when used with the value scramblers it passes strict PRNG tests. The generator is much faster than exsp_next/1 but with a bit lower quality and much shorter period.

mwc59_float(CX)

(since OTP 25.0)

-spec mwc59_float(CX :: mwc59_state()) -> V :: float().

Calculate a scrambled float/0 from a MWC59 state.

Returns a value V :: float/0 from a generator state CX, in the range 0.0 =< V < 1.0 like for example uniform_s/1.

The generator state is scrambled as with mwc59_value/1 before converted to a float/0.

mwc59_seed()

(since OTP 25.0)

-spec mwc59_seed() -> CX :: mwc59_state().

Create a MWC59 generator state.

Like mwc59_seed/1 but it hashes the default seed value of seed_s(atom()).

mwc59_seed(S)

(since OTP 25.0)

-spec mwc59_seed(S :: 0..1 bsl 58 - 1) -> CX :: mwc59_state().

Create a MWC59 generator state.

Returns a generator state CX. The 58-bit seed value S is hashed to create the generator state, to avoid that similar seeds create similar sequences.

mwc59_value32(CX)

(since OTP 25.0)

-spec mwc59_value32(CX :: mwc59_state()) -> V :: 0..1 bsl 32 - 1.

Calculate a 32-bit scrambled value from a MWC59 state.

Returns a 32-bit value V from a generator state CX. The generator state is scrambled using an 8-bit xorshift which masks the statistical imperfecions of the base generator mwc59 enough to produce numbers of decent quality. Still some problems in 2- and 3-dimensional birthday spacing and collision tests show through.

When using this scrambler it is in general better to use the high bits of the value than the low. The lowest 8 bits are of good quality and are passed right through from the base generator. They are combined with the next 8 in the xorshift making the low 16 good quality, but in the range 16..31 bits there are weaker bits that should not become high bits of the generated values.

Therefore it is in general safer to shift out low bits. See the recipes at the start of this Niche algorithms API description.

For a non power of 2 range less than about 16 bits (to not get too much bias and to avoid bignums) truncated multiplication can be used, that is: (Range*V) bsr 32, which is much faster than using rem.

mwc59_value(CX)

(since OTP 25.0)

-spec mwc59_value(CX :: mwc59_state()) -> V :: 0..1 bsl 59 - 1.

Calculate a 59-bit scrambled value from a MWC59 state.

Returns a 59-bit value V from a generator state CX. The generator state is scrambled using an 4-bit followed by a 27-bit xorshift, which masks the statistical imperfecions of the MWC59 base generator enough that all 59 bits are of very good quality.

Be careful to not accidentaly create a bignum when handling the value V.

It is in general general better to use the high bits from this scrambler than the low. See the recipes at the start of this Niche algorithms API description.

For a non power of 2 range less than about 29 bits (to not get too much bias and to avoid bignums) truncated multiplication can be used, which is much faster than using rem. Example for range 1'000'000'000; the range is 30 bits, we use 29 bits from the generator, adding up to 59 bits, which is not a bignum (on a 64-bit VM ): (1000000000 * (V bsr (59-29))) bsr 29.

splitmix64_next(AlgState)

(since OTP 25.0)

-spec splitmix64_next(AlgState :: integer()) -> {X :: uint64(), NewAlgState :: splitmix64_state()}.

Generate a SplitMix64 random 64-bit integer and new algorithm state.

From the specified AlgState generates a random 64-bit integer X and a new generator state NewAlgState, according to the SplitMix64 algorithm.

This generator is used internally in the rand module for seeding other generators since it is of a quite different breed which reduces the probability for creating an accidentally bad seed.

Plug-in framework API

bytes(N)

(since OTP 24.0)

-spec bytes(N :: non_neg_integer()) -> Bytes :: binary().

Generate random bytes as a t:binary(), using the state in the process dictionary.

Like bytes_s/2 but operates on the state stored in the process dictionary. Returns the generated Bytes.

bytes_s(N, State)

(since OTP 24.0)

-spec bytes_s(N :: non_neg_integer(), State :: state()) -> {Bytes :: binary(), NewState :: state()}.

Generate random bytes as a t:binary().

For a specified integer N >= 0, generates a binary/0 with that number of random bytes.

The selected algorithm is used to generate as many random numbers as required to compose the binary/0. Returns the generated Bytes and a NewState.

Note

The crypto module contains a function crypto:strong_rand_bytes/1 that does the same thing, but cryptographically secure. It is pretty fast and efficient on modern systems.

This function, however, offers the possibility to reproduce a byte sequence by re-using seed, which a cryptographically secure function cannot do.

Alas, because this function is based on a PRNG that produces random integers, thus has to create bytes from integers, it becomes rather slow.

Particularly inefficient and slow is to use a rand plug-in generator from crypto such as crypto:rand_seed_s/0 to call this function for generating bytes. Since in that case it is not possible to reproduce the byte sequence anyway; it is better to use crypto:strong_rand_bytes/1 directly.

export_seed()

(since OTP 18.0)

-spec export_seed() -> undefined | export_state().

Export the seed value.

Returns the random number state in an external format. To be used with seed/1.

export_seed_s(State)

(since OTP 18.0)

-spec export_seed_s(State :: state()) -> export_state().

Export the seed value.

Returns the random number generator state in an external format. To be used with seed/1.

jump()

(since OTP 20.0)

-spec jump() -> NewState :: state().

Jump the generator state forward.

Like jump/1 but operates on the state stored in the process dictionary. Returns the NewState.

jump(State)

(since OTP 20.0)

-spec jump(State :: state()) -> NewState :: state().

Jump the generator state forward.

Performs an algorithm specific State jump calculation that is equivalent to a large number of state iterations. See this module's algorithms list.

Returns the NewState.

This feature can be used to create many non-overlapping random number sequences from one start state; see the start of section Algorithms describing jump functions.

This function raises a not_implemented error exception if there is no jump function implemented for the State's algorithm.

normal()

(since OTP 18.0)

-spec normal() -> X :: float().

Generate a random number with standard normal distribution.

Like normal_s/1 but operates on the state stored in the process dictionary. Returns the generated number X.

normal(Mean, Variance)

(since OTP 20.0)

-spec normal(Mean :: number(), Variance :: number()) -> X :: float().

Generate a random number with specified normal distribution 𝒩 (μ, σ²).

Like normal_s/3 but operates on the state stored in the process dictionary. Returns the generated number X.

normal_s(State)

(since OTP 18.0)

-spec normal_s(State :: state()) -> {X :: float(), NewState :: state()}.

Generate a random number with standard normal distribution.

From the specified State, generates a random number X :: float/0, with standard normal distribution, that is with mean value 0.0 and variance 1.0.

Returns the generated number X and the NewState.

normal_s(Mean, Variance, State)

(since OTP 20.0)

-spec normal_s(Mean, Variance, State) -> {X :: float(), NewState :: state()}
                  when Mean :: number(), Variance :: number(), State :: state().

Generate a random number with specified normal distribution 𝒩 (μ, σ²).

From the specified State, generates a random number X :: float/0, with normal distribution 𝒩 (μ, σ²), that is 𝒩 (Mean, Variance) where Variance >= 0.0.

Returns X and the NewState.

seed(Alg_or_State)

(since OTP 18.0)

-spec seed(Alg | State) -> state()
              when Alg :: builtin_alg() | default, State :: state() | export_state().

Seed the random number generator and select algorithm.

The same as seed_s(Alg_or_State), but also stores the generated state in the process dictionary.

The argument default is an alias for the default algorithm that has been implemented (Since OTP 24.0).

seed(Alg, Seed)

(since OTP 18.0)

-spec seed(Alg, Seed) -> state() when Alg :: builtin_alg() | default, Seed :: seed().

Seed the random number generator and select algorithm.

The same as seed_s(Alg, Seed), but also stores the generated state in the process dictionary.

Alg = default is an alias for the default algorithm that has been implemented (Since OTP 24.0).

seed_s(Alg_or_State)

(since OTP 18.0)

-spec seed_s(Alg | State) -> state()
                when Alg :: builtin_alg() | default, State :: state() | export_state().

Seed the random number generator and select algorithm.

With the argument Alg, select that algorithm and seed random number generation with reasonably unpredictable time dependent data that should be unique to the created generator instance. It is (for now) based on the node name, the calling pid/0, the system time, and a system unique integer. This set of fairly unique items may change in the future, if necessary.

Alg = default is an alias for the default algorithm (Since OTP 24.0).

With the argument State, re-creates the state and returns it. See also export_seed/0.

seed_s(Alg, Seed)

(since OTP 18.0)

-spec seed_s(Alg, Seed) -> state() when Alg :: builtin_alg() | default, Seed :: seed().

Seed the random number generator and select algorithm.

Creates and returns a generator state for the specified algorithm from the specified seed/0 integers.

Alg = default is an alias for the default algorithm that has been implemented since OTP 24.0.

shuffle(List)

(since OTP 29.0)

-spec shuffle(List :: list()) -> ShuffledList :: list().

Shuffle a list.

Like shuffle_s/2 but operates on the state stored in the process dictionary. Returns the shuffled list.

shuffle_s(List, State)

(since OTP 29.0)

-spec shuffle_s(List, State) -> {ShuffledList :: list(), NewState :: state()}
                   when List :: list(), State :: state().

Shuffle a list.

From the specified State shuffles the elements in argument List so that, given that the PRNG algorithm in State is perfect, every possible permutation of the elements in the returned ShuffledList has the same probability.

In other words, the quality of the shuffling depends only on the quality of the backend random number generator and seed. If a cryptographically unpredictable shuffling is needed, use for example crypto:rand_seed_alg_s/1 to initialize the random number generator.

Returns the shuffled list ShuffledList and the NewState.

uniform()

(since OTP 18.0)

-spec uniform() -> X :: float().

Generate a uniformly distributed random number 0.0 =< X < 1.0, using the state in the process dictionary.

Like uniform_s/1 but operates on the state stored in the process dictionary. Returns the generated number X.

uniform(N)

(since OTP 18.0)

-spec uniform(N :: pos_integer()) -> X :: pos_integer().

Generate a uniformly distributed random integer 1 =< X =< N, using the state in the process dictionary.

Like uniform_s/2 but operates on the state stored in the process dictionary. Returns the generated number X.

uniform_real()

(since OTP 21.0)

-spec uniform_real() -> X :: float().

Generate a uniformly distributed random number 0.0 < X < 1.0, using the state in the process dictionary.

Like uniform_real_s/1 but operates on the state stored in the process dictionary. Returns the generated number X.

See uniform_real_s/1.

uniform_real_s(State)

(since OTP 21.0)

-spec uniform_real_s(State :: state()) -> {X :: float(), NewState :: state()}.

Generate a uniformly distributed random number 0.0 < X < 1.0.

From the specified state, generates a random float, uniformly distributed in the value range DBL_MIN =< X < 1.0.

Conceptually, a random real number R is generated from the interval 0.0 =< R < 1.0 and then the closest rounded down nonzero normalized number in the IEEE 754 Double Precision Format is returned.

Note

The generated numbers from this function has got better granularity for small numbers than the regular uniform_s/1 because all bits in the mantissa are random. This property, in combination with the fact that exactly zero is never returned is useful for algorithms doing for example 1.0 / X or math:log(X).

The concept implicates that the probability to get exactly zero is extremely low; so low that this function in fact never returns 0.0. The smallest number that it might return is DBL_MIN, which is 2.0^(-1022). However, the generators in this module have technical limitations on how many zero words in a row they can return, which limits the number of leading zeros that can be generated, which sets an upper limit for the smallest generated number, that is still extremely small.

The value range stated at the top of this function description is technically correct, but 0.0 =< X < 1.0 is a better description of the generated numbers' statistical distribution. That this function never returns exactly 0.0 is impossible to observe.

For all sub ranges N*2.0^(-53) =< X < (N+1)*2.0^(-53) where 0 =< integer(N) < 2.0^53, the probability to generate a number in a sub range is the same, very much like the numbers generated by uniform_s/1.

Having to generate extra random bits for occasional small numbers costs a little performance. This function is about 20% slower than the regular uniform_s/1

uniform_s(State)

(since OTP 18.0)

-spec uniform_s(State :: state()) -> {X :: float(), NewState :: state()}.

Generate a uniformly distributed random number 0.0 =< X < 1.0.

From the specified State, generates a random number X :: float/0, uniformly distributed in the value range 0.0 =< X < 1.0. Returns the number X and the updated NewState.

The generated numbers are of the form N * 2.0^(-53), that is; equally spaced in the interval.

Warning

This function may return exactly 0.0 which can be fatal for certain applications. If that is undesired you can use (1.0 - rand:uniform()) to get the interval 0.0 < X =< 1.0, or instead use uniform_real/0.

If neither endpoint is desired you can achieve the range 0.0 < X < 1.0 using test and re-try like this:

my_uniform() ->
    case rand:uniform() of
        X when 0.0 < X -> X;
        _ -> my_uniform()
    end.

uniform_s(N, State)

(since OTP 18.0)

-spec uniform_s(N :: pos_integer(), State :: state()) -> {X :: pos_integer(), NewState :: state()}.

Generate a uniformly distributed random integer 1 =< X =< N.

From the specified State, generates a random number X :: integer/0, uniformly distributed in the specified range 1 =< X =< N. Returns the number X and the updated NewState.

random

This module is deprecated. Use the module 'rand' instead.

Pseudo-random number generation.

This module provides a random number generator. The method is attributed to B.A. Wichmann and I.D. Hill in 'An efficient and portable pseudo-random number generator', Journal of Applied Statistics. AS183. 1982. Also Byte March 1987.

The algorithm is a modification of the version attributed to Richard A. O'Keefe in the standard Prolog library.

Every time a random number is requested, a state is used to calculate it, and a new state is produced. The state can either be implicit (kept in the process dictionary) or be an explicit argument and return value. In this implementation, the state (the type ran/0) consists of a tuple of three integers.

Note

This random number generator is not cryptographically strong. If a strong cryptographic random number generator is needed, use one of functions in the crypto module, for example, crypto:strong_rand_bytes/1.

Note

The improved rand module is to be used instead of this module.

Note

Some of the functions use the process dictionary variable random_seed to remember the current seed.

If a process calls uniform/0 or uniform/1 without setting a seed first, seed/0 is called automatically.

The implementation changed in Erlang/OTP R15. Upgrading to R15 breaks applications that expect a specific output for a specified seed. The output is still deterministic number series, but different compared to releases older than R15. Seed {0,0,0} does, for example, no longer produce a flawed series of only zeros.

Summary

Types

ran()

The state.

Functions

seed0() deprecated

Returns the default state.

seed() deprecated

Seeds random number generation with default (fixed) values in the process dictionary and returns the old state.

seed(SValue) deprecated

seed({A1, A2, A3}) is equivalent to seed(A1, A2, A3).

seed(A1, A2, A3) deprecated

Seeds random number generation with integer values in the process dictionary and returns the old state.

uniform() deprecated

Returns a random float uniformly distributed between 0.0 and 1.0, updating the state in the process dictionary.

uniform(N) deprecated

Returns, for a specified integer N >= 1, a random integer uniformly distributed between 1 and N, updating the state in the process dictionary.

uniform_s(State0) deprecated

Returns, for a specified state, a random float uniformly distributed between 0.0 and 1.0, and a new state.

uniform_s(N, State0) deprecated

Returns, for a specified integer N >= 1 and a state, a random integer uniformly distributed between 1 and N, and a new state.

Types

ran()

(not exported)

-type ran() :: {integer(), integer(), integer()}.

The state.

Functions

seed0()

This function is deprecated. random:seed0/0 is deprecated; use the 'rand' module instead.

-spec seed0() -> ran().

Returns the default state.

seed()

This function is deprecated. random:seed/0 is deprecated; use the 'rand' module instead.

-spec seed() -> ran().

Seeds random number generation with default (fixed) values in the process dictionary and returns the old state.

seed(SValue)

This function is deprecated. random:seed/1 is deprecated; use the 'rand' module instead.

-spec seed(SValue) -> undefined | ran()
              when SValue :: {A1, A2, A3} | integer(), A1 :: integer(), A2 :: integer(), A3 :: integer().

seed({A1, A2, A3}) is equivalent to seed(A1, A2, A3).

seed(A1, A2, A3)

This function is deprecated. random:seed/3 is deprecated; use the 'rand' module instead.

-spec seed(A1, A2, A3) -> undefined | ran() when A1 :: integer(), A2 :: integer(), A3 :: integer().

Seeds random number generation with integer values in the process dictionary and returns the old state.

The following is an easy way of obtaining a unique value to seed with:

random:seed(erlang:phash2([node()]),
            erlang:monotonic_time(),
            erlang:unique_integer())

For details, see erlang:phash2/1, erlang:node/0, erlang:monotonic_time/0, and erlang:unique_integer/0.

uniform()

This function is deprecated. random:uniform/0 is deprecated; use the 'rand' module instead.

-spec uniform() -> float().

Returns a random float uniformly distributed between 0.0 and 1.0, updating the state in the process dictionary.

uniform(N)

This function is deprecated. random:uniform/1 is deprecated; use the 'rand' module instead.

-spec uniform(N) -> pos_integer() when N :: pos_integer().

Returns, for a specified integer N >= 1, a random integer uniformly distributed between 1 and N, updating the state in the process dictionary.

uniform_s(State0)

This function is deprecated. random:uniform_s/1 is deprecated; use the 'rand' module instead.

-spec uniform_s(State0) -> {float(), State1} when State0 :: ran(), State1 :: ran().

Returns, for a specified state, a random float uniformly distributed between 0.0 and 1.0, and a new state.

uniform_s(N, State0)

This function is deprecated. random:uniform_s/2 is deprecated; use the 'rand' module instead.

-spec uniform_s(N, State0) -> {integer(), State1}
                   when N :: pos_integer(), State0 :: ran(), State1 :: ran().

Returns, for a specified integer N >= 1 and a state, a random integer uniformly distributed between 1 and N, and a new state.

zip

Utility for reading and creating 'zip' archives.

This module archives and extracts files to and from a zip archive. The zip format is specified by the "ZIP Appnote.txt" file, available on the PKWARE web site www.pkware.com.

The zip module supports zip archive versions up to 6.1. However, password-protection is not supported.

By convention, the name of a zip file is to end with .zip. To abide to the convention, add .zip to the filename.

To create zip archives, use function zip/2 or zip/3. They are also available as create/2,3, to resemble the erl_tar module.

To extract files from a zip archive, use function unzip/1 or unzip/2. They are also available as extract/1,2, to resemble the erl_tar module.

To fold a function over all files in a zip archive, use function foldl/3.

To return a list of the files in a zip archive, use function list_dir/1 or list_dir/2. They are also available as table/1,2, to resemble the erl_tar module.

To print a list of files to the Erlang shell, use function t/1 or tt/1.

Sometimes it is desirable to open a zip archive, and to unzip files from it file by file, without having to reopen the archive. This can be done by functions zip_open/1,2, zip_get/1,2, zip_list_dir/1, and zip_close/1.

The ZIP extensions 0x5355 "extended timestamps" and 0x7875 "UID+GID handling" are supported. Both extensions are by default enabled when creating an archive, but only "extended timestamps" are enabled when extracting. Use the extra/0 option to change how these extensions are used.

Limitations

Password-protected and encrypted archives are not supported.

Only the DEFLATE (zlib-compression) and the STORE (uncompressed data) zip methods are supported.

Comments for individual files are not supported when creating zip archives. The zip archive comment for the whole zip archive is supported.

Changing a zip archive is not supported. To add or remove a file from an archive, the whole archive must be recreated.

Summary

Types

create_option()

These options are described in create/3.

extension()

A filename extension, for example ".txt".

extension_spec()

extra()

The possible extra extension that can be used.

filename()

The name of a zip file.

handle()

As returned by zip_open/2.

zip_comment()

The record zip_comment only contains the archive comment for a zip archive.

zip_file()

The record zip_file contains the following fields

Functions

foldl(Fun, Acc0, Archive)

Calls Fun(FileInArchive, GetInfo, GetBin, AccIn) on successive files in the Archive, starting with AccIn == Acc0.

list_dir(Archive)

Equivalent to list_dir(Archive, []).

list_dir(Archive, Options)

Retrieves all filenames in the zip archive Archive.

t(Archive)

Prints all filenames in the zip archive Archive to the Erlang shell. (Similar to tar t.)

tt(Archive)

Prints filenames and information about all files in the zip archive Archive to the Erlang shell. (Similar to tar tv.)

unzip(Archive)

Equivalent to unzip(Archive, []).

unzip(Archive, Options)

Extracts all files from a zip archive.

zip(Name, FileList)

Equivalent to zip(Name, FileList, []).

zip(Name, FileList, Options)

Creates a zip archive containing the files specified in FileList.

zip_close(ZipHandle)

Closes a zip archive, previously opened with zip_open/1,2. All resources are closed, and the handle is not to be used after closing.

zip_get(ZipHandle)

Equivalent to zip_get/2.

zip_get(FileName, ZipHandle)

Extracts one or all files from an open archive.

zip_get_crc32(FileName, ZipHandle)

Extracts one crc32 checksum from an open archive.

zip_list_dir(ZipHandle)

Returns the file list of an open zip archive. The first returned element is the zip archive comment.

zip_open(Archive)

Equivalent to zip_open/2.

zip_open(Archive, Options)

Opens a zip archive, and reads and saves its directory. This means that later reading files from the archive is faster than unzipping files one at a time with unzip/1,2.

erl_tar compatibility functions

create(Name, FileList)

Equivalent to zip(Name, FileList).

create(Name, FileList, Options)

Equivalent to zip(Name, FileList, Options).

extract(Archive)

Equivalent to unzip(Archive).

extract(Archive, Options)

Equivalent to unzip(Archive, Options).

table(Archive)

Equivalent to list_dir(Archive, []).

table(Archive, Options)

Equivalent to list_dir(Archive, Options).

Types

create_option()

-type create_option() ::
          memory | cooked | verbose |
          {comment, Comment :: string()} |
          {cwd, CWD :: file:filename()} |
          {compress, What :: extension_spec()} |
          {uncompress, What :: extension_spec()} |
          {extra, extra()}.

These options are described in create/3.

extension()

(not exported)

-type extension() :: string().

A filename extension, for example ".txt".

extension_spec()

(not exported)

-type extension_spec() ::
          all |
          [Extension :: extension()] |
          {add, [Extension :: extension()]} |
          {del, [Extension :: extension()]}.

extra()

(not exported)

-type extra() :: [extended_timestamp | uid_gid].

The possible extra extension that can be used.

extended_timestamp - enables the 0x5455 "extended timestamps" zip extension that embeds POSIX timestamps for access and modification times for each file in the archive. This makes the timestamps to be in UTC instead of local time and also increases the time resolution from 2 seconds to 1 second.

uid_gid - enables 0x7875 "UNIX 3rd generation" zip extension that embeds the UID and GID for each file into the archive.

filename()

-type filename() :: file:filename().

The name of a zip file.

handle()

-opaque handle()

As returned by zip_open/2.

zip_comment()

(not exported)

-type zip_comment() :: #zip_comment{comment :: string()}.

The record zip_comment only contains the archive comment for a zip archive.

zip_file()

(not exported)

-type zip_file() ::
          #zip_file{name :: string(),
                    info :: file:file_info(),
                    comment :: string(),
                    offset :: non_neg_integer(),
                    comp_size :: non_neg_integer()}.

The record zip_file contains the following fields:

name - The filename

info - File information as in file:read_file_info/1 in Kernel. mtime, atime and ctime are expected to be in local time if represented using calendar:datetime/0, or in OS system time if represented by an integer.

comment - The comment for the file in the zip archive

offset - The file offset in the zip archive (used internally)

comp_size - The size of the compressed file (the size of the uncompressed file is found in info)

Functions

foldl(Fun, Acc0, Archive)

(since OTP R14B)

-spec foldl(Fun, Acc0, Archive) -> {ok, Acc1} | {error, Reason}
               when
                   Fun :: fun((FileInArchive, GetInfo, GetBin, AccIn) -> AccOut),
                   FileInArchive :: file:name(),
                   GetInfo :: fun(() -> file:file_info()),
                   GetBin :: fun(() -> binary()),
                   Acc0 :: term(),
                   Acc1 :: term(),
                   AccIn :: term(),
                   AccOut :: term(),
                   Archive :: file:name() | {file:name(), binary()},
                   Reason :: term().

Calls Fun(FileInArchive, GetInfo, GetBin, AccIn) on successive files in the Archive, starting with AccIn == Acc0.

FileInArchive is the name that the file has in the archive.

GetInfo is a fun that returns information about the file.

GetBin returns the file contents.

Both GetInfo and GetBin must be called within the Fun. Their behavior is undefined if they are called outside the context of Fun.

The Fun must return a new accumulator, which is passed to the next call. foldl/3 returns the final accumulator value. Acc0 is returned if the archive is empty. It is not necessary to iterate over all files in the archive. The iteration can be ended prematurely in a controlled manner by throwing an exception.

Example:

> Name = "dummy.zip".
"dummy.zip"
> {ok, {Name, Bin}} = zip:create(Name, [{"foo", <<"FOO">>}, {"bar", <<"BAR">>}], [memory]).
{ok,{"dummy.zip",
     <<80,75,3,4,20,0,0,0,0,0,74,152,97,60,171,39,212,26,3,0,
       0,0,3,0,0,...>>}}
> {ok, FileSpec} = zip:foldl(fun(N, I, B, Acc) -> [{N, B(), I()} | Acc] end, [], {Name, Bin}).
{ok,[{"bar",<<"BAR">>,
      {file_info,3,regular,read_write,
                 {{2010,3,1},{19,2,10}},
                 {{2010,3,1},{19,2,10}},
                 {{2010,3,1},{19,2,10}},
                 54,1,0,0,0,0,0}},
     {"foo",<<"FOO">>,
      {file_info,3,regular,read_write,
                 {{2010,3,1},{19,2,10}},
                 {{2010,3,1},{19,2,10}},
                 {{2010,3,1},{19,2,10}},
                 54,1,0,0,0,0,0}}]}
> {ok, {Name, Bin}} = zip:create(Name, lists:reverse(FileSpec), [memory]).
{ok,{"dummy.zip",
     <<80,75,3,4,20,0,0,0,0,0,74,152,97,60,171,39,212,26,3,0,
       0,0,3,0,0,...>>}}
> catch zip:foldl(fun("foo", _, B, _) -> throw(B()); (_,_,_,Acc) -> Acc end, [], {Name, Bin}).
<<"FOO">>

list_dir(Archive)

-spec list_dir(Archive) -> RetValue
                  when
                      Archive :: file:name() | binary(),
                      RetValue :: {ok, CommentAndFiles} | {error, Reason :: term()},
                      CommentAndFiles :: [zip_comment() | zip_file()].

Equivalent to list_dir(Archive, []).

list_dir(Archive, Options)

-spec list_dir(Archive, Options) -> RetValue
                  when
                      Archive :: file:name() | binary(),
                      RetValue :: {ok, CommentAndFiles} | {error, Reason :: term()},
                      CommentAndFiles :: [zip_comment() | zip_file()],
                      Options :: [Option],
                      Option :: cooked | skip_directories | {extra, extra()}.

Retrieves all filenames in the zip archive Archive.

The result value is the tuple {ok, List}, where List contains the zip archive comment as the first element.

One option is available:

cooked - By default, this function opens the zip file in raw mode, which is faster but does not allow a remote (Erlang) file server to be used. Adding cooked to the mode list overrides the default and opens the zip file without option raw.

skip_directories - By default empty directories within zip archives are listed. With option skip_directories set, empty directories are no longer listed.

{extra, Extras} - The zip "extra" features to respect. The supported "extra" features are "extended timestamps" and "UID and GID" handling. By default only "extended timestamps" is enabled when listing files. See extra/0 for more details.

t(Archive)

-spec t(Archive) -> ok when Archive :: file:name() | binary() | ZipHandle, ZipHandle :: handle().

Prints all filenames in the zip archive Archive to the Erlang shell. (Similar to tar t.)

tt(Archive)

-spec tt(Archive) -> ok when Archive :: file:name() | binary() | ZipHandle, ZipHandle :: handle().

Prints filenames and information about all files in the zip archive Archive to the Erlang shell. (Similar to tar tv.)

unzip(Archive)

-spec unzip(Archive) -> RetValue
               when
                   Archive :: file:name() | binary(),
                   RetValue ::
                       {ok, FileList} |
                       {ok, FileBinList} |
                       {error, Reason :: term()} |
                       {error, {Name :: file:name(), Reason :: term()}},
                   FileList :: [file:name()],
                   FileBinList :: [{file:name(), binary()}].

Equivalent to unzip(Archive, []).

unzip(Archive, Options)

-spec unzip(Archive, Options) -> RetValue
               when
                   Archive :: file:name() | binary(),
                   Options :: [Option],
                   Option ::
                       {file_list, FileList} |
                       cooked | keep_old_files | verbose | memory | skip_directories |
                       {file_filter, FileFilter} |
                       {cwd, CWD} |
                       {extra, extra()},
                   FileList :: [file:name()],
                   FileBinList :: [{file:name(), binary()}],
                   FileFilter :: fun((ZipFile) -> boolean()),
                   CWD :: file:filename(),
                   ZipFile :: zip_file(),
                   RetValue ::
                       {ok, FileList} |
                       {ok, FileBinList} |
                       {error, Reason :: term()} |
                       {error, {Name :: file:name(), Reason :: term()}}.

Extracts all files from a zip archive.

If argument Archive is specified as a binary/0, the contents of the binary is assumed to be a zip archive, otherwise a filename.

Options:

{file_list, FileList} - By default, all files are extracted from the zip archive. With option {file_list, FileList}, function unzip/2 only extracts the files whose names are included in FileList. The full paths, including the names of all subdirectories within the zip archive, must be specified.

cooked - By default, this function opens the zip file in raw mode, which is faster but does not allow a remote (Erlang) file server to be used. Adding cooked to the mode list overrides the default and opens the zip file without option raw. The same applies for the files extracted.

keep_old_files - By default, all files with the same name as files in the zip archive are overwritten. With option keep_old_files set, function unzip/2 does not overwrite existing files. Notice that even with option memory specified, which means that no files are overwritten, existing files are excluded from the result.

skip_directories - By default empty directories within zip archives are extracted. With option skip_directories set, empty directories are no longer created.

{extra, Extras} - The zip "extra" features to respect. The supported "extra" features are "extended timestamps" and "UID and GID" handling. By default only "extended timestamps" is enabled when unzipping. See extra/0 for more details.

verbose - Prints an informational message for each extracted file.

memory - Instead of extracting to the current directory, the result is given as a list of tuples {Filename, Binary}, where Binary is a binary containing the extracted data of file Filename in the zip archive.

{cwd, CWD} - Uses the specified directory as current directory. It is prepended to filenames when extracting them from the zip archive. (Acting like file:set_cwd/1 in Kernel, but without changing the global cwd property.)

zip(Name, FileList)

-spec zip(Name, FileList) -> RetValue
             when
                 Name :: file:name(),
                 FileList :: [FileSpec],
                 FileSpec ::
                     file:name() | {file:name(), binary()} | {file:name(), binary(), file:file_info()},
                 RetValue ::
                     {ok, FileName :: file:name()} |
                     {ok, {FileName :: file:name(), binary()}} |
                     {error, Reason :: term()}.

Equivalent to zip(Name, FileList, []).

zip(Name, FileList, Options)

-spec zip(Name, FileList, Options) -> RetValue
             when
                 Name :: file:name(),
                 FileList :: [FileSpec],
                 FileSpec ::
                     file:name() | {file:name(), binary()} | {file:name(), binary(), file:file_info()},
                 Options :: [Option],
                 Option :: create_option(),
                 RetValue ::
                     {ok, FileName :: file:name()} |
                     {ok, {FileName :: file:name(), binary()}} |
                     {error, Reason :: term()}.

Creates a zip archive containing the files specified in FileList.

FileList is a list of files, with paths relative to the current directory, which are stored with this path in the archive. File system operations are performed to read the file metadata and, when compression is enabled, to stream the file contents without loading whole files into memory. Files can also be specified as binaries to create an archive directly from data. In such cases, no metadata or file system reads are performed.

Files are compressed using the DEFLATE compression, as described in the "Appnote.txt" file. However, files are stored without compression if they are already compressed. zip/2 and zip/3 check the file extension to determine if the file is to be stored without compression. Files with the following extensions are not compressed: .Z, .zip, .zoo, .arc, .lzh, .arj.

It is possible to override the default behavior and control what types of files that are to be compressed by using options {compress, What} and {uncompress, What}. It is also possible to use many compress and uncompress options.

To trigger file compression, its extension must match with the compress condition and must not match the uncompress condition. For example, if compress is set to ["gif", "jpg"] and uncompress is set to ["jpg"], only files with extension "gif" are compressed.

Options:

cooked - By default, this function opens the zip file in mode raw, which is faster but does not allow a remote (Erlang) file server to be used. Adding cooked to the mode list overrides the default and opens the zip file without the raw option. The same applies for the files added.

verbose - Prints an informational message about each added file.

memory - The output is not to a file, but instead as a tuple {FileName, binary()}. The binary is a full zip archive with header and can be extracted with, for example, unzip/2.

{comment, Comment} - Adds a comment to the zip archive.

{cwd, CWD} - Uses the specified directory as current work directory (cwd). This is prepended to filenames when adding them, although not in the zip archive (acting like file:set_cwd/1 in Kernel, but without changing the global cwd property.).

{extra, Extras} - The zip "extra" features to respect. The supported "extra" features are "extended timestamps" and "UID and GID" handling. By default both these "extra" features are enabled. See extra/0 for more details.

{compress, What} - Controls what types of files to be compressed. Defaults to all. The following values of What are allowed:

all - All files are compressed (as long as they pass the uncompress condition).

[Extension] - Only files with exactly these extensions are compressed.

{add,[Extension]} - Adds these extensions to the list of compress extensions.

{del,[Extension]} - Deletes these extensions from the list of compress extensions.

{uncompress, What} - Controls what types of files to be uncompressed. Defaults to [".Z", ".zip", ".zoo", ".arc", ".lzh", ".arj"]. The following values of What are allowed:

all - No files are compressed.

[Extension] - Files with these extensions are uncompressed.

{add,[Extension]} - Adds these extensions to the list of uncompress extensions.

{del,[Extension]} - Deletes these extensions from the list of uncompress extensions.

zip_close(ZipHandle)

-spec zip_close(ZipHandle) -> ok | {error, einval} when ZipHandle :: handle().

Closes a zip archive, previously opened with zip_open/1,2. All resources are closed, and the handle is not to be used after closing.

zip_get(ZipHandle)

-spec zip_get(ZipHandle) -> {ok, [Result]} | {error, Reason}
                 when
                     ZipHandle :: handle(),
                     Result :: file:name() | {file:name(), binary()},
                     Reason :: term().

Equivalent to zip_get/2.

zip_get(FileName, ZipHandle)

-spec zip_get(FileName, ZipHandle) -> {ok, Result} | {error, Reason}
                 when
                     FileName :: file:name(),
                     ZipHandle :: handle(),
                     Result :: file:name() | {file:name(), binary()},
                     Reason :: term().

Extracts one or all files from an open archive.

The files are unzipped to memory or to file, depending on the options specified to function zip_open/1,2 when opening the archive.

zip_get_crc32(FileName, ZipHandle)

(since OTP 26.0)

-spec zip_get_crc32(FileName, ZipHandle) -> {ok, CRC} | {error, Reason}
                       when
                           FileName :: file:name(),
                           ZipHandle :: handle(),
                           CRC :: non_neg_integer(),
                           Reason :: term().

Extracts one crc32 checksum from an open archive.

zip_list_dir(ZipHandle)

-spec zip_list_dir(ZipHandle) -> {ok, Result} | {error, Reason}
                      when
                          Result :: [zip_comment() | zip_file()],
                          ZipHandle :: handle(),
                          Reason :: term().

Returns the file list of an open zip archive. The first returned element is the zip archive comment.

zip_open(Archive)

-spec zip_open(Archive) -> {ok, ZipHandle} | {error, Reason}
                  when Archive :: file:name() | binary(), ZipHandle :: handle(), Reason :: term().

Equivalent to zip_open/2.

zip_open(Archive, Options)

-spec zip_open(Archive, Options) -> {ok, ZipHandle} | {error, Reason}
                  when
                      Archive :: file:name() | binary(),
                      ZipHandle :: handle(),
                      Options :: [Option],
                      Option :: cooked | memory | {cwd, CWD :: file:filename()} | {extra, extra()},
                      Reason :: term().

Opens a zip archive, and reads and saves its directory. This means that later reading files from the archive is faster than unzipping files one at a time with unzip/1,2.

The options are equivalent to those in unzip/2.

The archive must be closed with zip_close/1.

The ZipHandle is closed if the process that originally opened the archive dies.

erl_tar compatibility functions

create(Name, FileList)

-spec create(Name, FileList) -> RetValue
                when
                    Name :: file:name(),
                    FileList :: [FileSpec],
                    FileSpec ::
                        file:name() |
                        {file:name(), binary()} |
                        {file:name(), binary(), file:file_info()},
                    RetValue ::
                        {ok, FileName :: filename()} |
                        {ok, {FileName :: filename(), binary()}} |
                        {error, Reason :: term()}.

Equivalent to zip(Name, FileList).

create(Name, FileList, Options)

-spec create(Name, FileList, Options) -> RetValue
                when
                    Name :: file:name(),
                    FileList :: [FileSpec],
                    FileSpec ::
                        file:name() |
                        {file:name(), binary()} |
                        {file:name(), binary(), file:file_info()},
                    Options :: [Option],
                    Option :: create_option(),
                    RetValue ::
                        {ok, FileName :: filename()} |
                        {ok, {FileName :: filename(), binary()}} |
                        {error, Reason :: term()}.

Equivalent to zip(Name, FileList, Options).

extract(Archive)

-spec extract(Archive) -> RetValue
                 when
                     Archive :: file:name() | binary(),
                     RetValue ::
                         {ok, FileList} |
                         {ok, FileBinList} |
                         {error, Reason :: term()} |
                         {error, {Name :: file:name(), Reason :: term()}},
                     FileList :: [file:name()],
                     FileBinList :: [{file:name(), binary()}].

Equivalent to unzip(Archive).

extract(Archive, Options)

-spec extract(Archive, Options) -> RetValue
                 when
                     Archive :: file:name() | binary(),
                     Options :: [Option],
                     Option ::
                         {file_list, FileList} |
                         keep_old_files | verbose | memory |
                         {file_filter, FileFilter} |
                         {cwd, CWD},
                     FileList :: [file:name()],
                     FileBinList :: [{file:name(), binary()}],
                     FileFilter :: fun((ZipFile) -> boolean()),
                     CWD :: file:filename(),
                     ZipFile :: zip_file(),
                     RetValue ::
                         {ok, FileList} |
                         {ok, FileBinList} |
                         {error, Reason :: term()} |
                         {error, {Name :: file:name(), Reason :: term()}}.

Equivalent to unzip(Archive, Options).

table(Archive)

-spec table(Archive) -> RetValue
               when
                   Archive :: file:name() | binary(),
                   RetValue :: {ok, CommentAndFiles} | {error, Reason :: term()},
                   CommentAndFiles :: [zip_comment() | zip_file()].

Equivalent to list_dir(Archive, []).

table(Archive, Options)

-spec table(Archive, Options) -> RetValue
               when
                   Archive :: file:name() | binary(),
                   RetValue :: {ok, CommentAndFiles} | {error, Reason :: term()},
                   CommentAndFiles :: [zip_comment() | zip_file()],
                   Options :: [Option],
                   Option :: cooked.

Equivalent to list_dir(Archive, Options).

zstd

Zstandard compression interface.

This module provides an API for the Zstandard library (www.zstd.net). It is used to compress and decompress data and offers the same compression ratio as zlib but at a lower CPU cost.

Example:

1> Data = ~"my data to be compressed".
2> Compressed = zstd:compress(Data).
3> zstd:decompress(Compressed).
[~"my data to be compressed"]

If you are compressing or decompressing possibly large amounts of data, it is also possible to do streamed compression/decompression.

Example:

1> Compress = fun F(Ctx, D) ->
                      case file:read(D, 5) of
                          {ok, Data} ->
                              {continue, C} = zstd:stream(Ctx, Data),
                              [C|F(Ctx, D)];
                          eof ->
                              {done, C} = zstd:finish(Ctx, ""),
                              C
                      end
              end.
2> {ok, Ctx} = zstd:context(compress).
3> {ok, D} = file:open(File,[read,binary]).
4> Compressed = iolist_to_binary(Compress(Ctx, D)).
<<40,181,47,253,0,88,89,0,0,108,111,114,101,109,32,105,112,115,117,109>>
5> zstd:decompress(Compressed).
[~"lorem ipsum"]

In all functions errors can be thrown, where Reason describes the error.

Typical Reasons:

badarg - Bad argument.

zstd_error - An error generated by the Zstandard library.

not_on_controlling_process - The context was used by a process that did not create it.

Summary

Types

compress_parameters()

Compression parameters.

compressionLevel()

The compression level.

context()

A compression or decompression context that can be used for streaming compression or decompression.

decompress_parameters()

Decompression parameters.

dict()

A compression or decompression dictionary.

strategy()

The compression strategy.

Functions

close(Ctx)

Close a context/0, releasing all referenced resources. After a context/0 is closed it is no longer possible to use it.

compress(Data)

Equivalent to compress(Data, #{}).

compress(Data, CtxOrOptions)

Compress Data using the given compress_parameters/0 or the context/0.

context(Mode)

Equivalent to context(Mode, #{}).

context(Mode, Options)

Create a compression or decompression context.

decompress(Data)

Equivalent to decompress(Data, #{}).

decompress(Data, CtxOrOptions)

Decompress Data using the given decompress_parameters/0 or the context/0.

dict(Mode, Dict)

Equivalent to dict(Mode, Dict, #{}).

dict/3

Create a compression or decompression dictionary.

finish(Ctx, Data)

Finish compressing/decompressing data.

get_dict_id(DictOrFrame)

Get the dictionary ID of a dictionary or a frame.

get_frame_header(Frame)

Get header of a Zstandard compressed frame.

get_parameter(Ctx, Key)

Get a parameter from a context/0.

reset(Ctx)

Reset a context while streaming data, returning it to its original state but keeping all parameters set.

set_parameter(Ctx, Key, Value)

Set a parameter on a context/0.

stream(Ctx, Data)

Compress or decompress a stream of data. The last stream of data should be called with finish/2 to complete the compression/decompression.

Types

compress_parameters()

(not exported) (since OTP 28.0)

-type compress_parameters() ::
          #{dictionary => binary() | dict(),
            pledgedSrcSize => non_neg_integer(),
            compressionLevel => compressionLevel(),
            windowLog => non_neg_integer(),
            hashLog => non_neg_integer(),
            chainLog => non_neg_integer(),
            searchLog => non_neg_integer(),
            minMatch => non_neg_integer(),
            targetLength => non_neg_integer(),
            strategy => strategy(),
            targetCBlockSize => non_neg_integer(),
            enableLongDistanceMatching => boolean(),
            ldmHashLog => non_neg_integer(),
            ldmMinMatch => non_neg_integer(),
            ldmBucketSizeLog => non_neg_integer(),
            ldmHashRateLog => non_neg_integer(),
            contentSizeFlag => boolean(),
            checksumFlag => boolean(),
            dictIDFlag => boolean()}.

Compression parameters.

Zstandard has many parameters that can be tuned. Setting some parameters will fail when set to an incorrect value, while others will be silently adjusted to the closest valid value.

dictionary - Sets the compression dictionary for the context. The dictionary can be either a t:binary() representing the dictionary or a compression dict/0. When a dict/0 is attached to a context it will be kept alive until either the context is closed or it is replaced by another dictionary.

To reset the context to not use any dictionary use the empty dictionary, that is <<>>.

pledgedSrcSize - When using stream/2 to do streaming compression, the decompressed size is not known when the header of the Zstandard frame is emitted. Setting this parameter on the context lets the compressor know the expected size of the data to compress. If the size is not correct when finish/2 is called an exception will be generated. Using compress/1,2 will automatically set this value.

compressionLevel - Sets the compressionLevel/0.

windowLog | hashLog | chainLog | searchLog | minMatch | targetLength | targetCBlockSize - Set the corresponding parameter. See the Zstandard documentation for more details.

strategy - Sets the compression strategy/0.

enableLongDistanceMatching - Whether to enable Long Distance Matching or not. LDM is useful when compressing large datasets and is enabled by using a high compressionLevel.

ldmHashLog | ldmMinMatch | ldmBucketSizeLog | ldmHashRateLog - Set the corresponding LDM parameter. See the Zstandard documentation for more details.

contentSizeFlag - Whether to include the contentSize or not.

checksumFlag - Whether to include the checksum or not.

dictIDFlag - Whether to include the dictionary ID or not.

The Zstandard documentation contains more details about each parameter.

compressionLevel()

(not exported) (since OTP 28.0)

-type compressionLevel() :: -22..22.

The compression level.

Higher values mean better compression ratio at the sacrifice of performance. A negative value sacrifices compression ratio in favor of performance.

0 is a special value which represents the default compression level.

context()

(since OTP 28.0)

-opaque context()

A compression or decompression context that can be used for streaming compression or decompression.

Only the process that created the context can use it.

decompress_parameters()

(not exported) (since OTP 28.0)

-type decompress_parameters() :: #{dictionary => binary() | dict(), windowLogMax => non_neg_integer()}.

Decompression parameters.

Zstandard has many parameters that can be tuned. Setting some parameters will fail when set to an incorrect value, while others will be silently adjusted to the closest valid value.

dictionary - Sets the decompression dictionary for the context. The dictionary can be either a t:binary() representing the dictionary or a decompression dict/0. When a dict/0 is attached to a context it will be kept alive until either the context is closed or it is replaced by another dictionary.

To reset the context to not use any dictionary use the empty dictionary, that is <<>>.

windowLogMax - Set the corresponding parameter. See the Zstandard documentation for more details.

dict()

(since OTP 28.0)

-opaque dict()

A compression or decompression dictionary.

strategy()

(not exported) (since OTP 28.0)

-type strategy() ::
          default | fast | dfast | greedy | lazy | lazy2 | btlazy2 | btopt | btultra | btultra2.

The compression strategy.

The strategies are listed depending on which compression ratio they give, that is the fast strategy is the fastest but also has the worst compression ratio, while btultra2 is the slowest but has the best compression ratio.

default is a special strategy representing the current default strategy.

See the Zstandard documentation for details on each strategy.

Functions

close(Ctx)

(since OTP 28.0)

-spec close(Ctx :: context()) -> ok.

Close a context/0, releasing all referenced resources. After a context/0 is closed it is no longer possible to use it.

A context/0 is automatically closed when GC:ed, so the only reason to call this function is to make the resources attached to the context be released before the next GC.

compress(Data)

(since OTP 28.0)

-spec compress(iodata()) -> iodata().

Equivalent to compress(Data, #{}).

compress(Data, CtxOrOptions)

(since OTP 28.0)

-spec compress(Data :: iodata(), Options :: compress_parameters()) -> iodata();
              (Data :: iodata(), Ctx :: context()) -> iodata().

Compress Data using the given compress_parameters/0 or the context/0.

Example:

1> zstd:compress("abc").
2> zstd:compress("abc", #{ compressionLevel => 20 }).

context(Mode)

(since OTP 28.0)

-spec context(compress | decompress) -> {ok, context()}.

Equivalent to context(Mode, #{}).

context(Mode, Options)

(since OTP 28.0)

-spec context(compress, Options :: compress_parameters()) -> {ok, context()};
             (decompress, Options :: decompress_parameters()) -> {ok, context()}.

Create a compression or decompression context.

A context can be used to do streaming compression/decompression and allows re-using parameters for multiple compressions/decompressions.

decompress(Data)

(since OTP 28.0)

-spec decompress(iodata()) -> iodata().

Equivalent to decompress(Data, #{}).

decompress(Data, CtxOrOptions)

(since OTP 28.0)

-spec decompress(Data :: iodata(), Options :: decompress_parameters()) -> iodata();
                (Data :: iodata(), Ctx :: context()) -> iodata().

Decompress Data using the given decompress_parameters/0 or the context/0.

Example:

1> Compressed = zstd:compress("abc").
2> zstd:decompress(Compressed).
[~"abc"]

dict(Mode, Dict)

(since OTP 28.0)

-spec dict(Mode :: compress | decompress, Dict :: binary()) -> {ok, dict()}.

Equivalent to dict(Mode, Dict, #{}).

dict/3

(since OTP 28.0)

-spec dict(compress, Dict :: binary(), #{compressionLevel => compressionLevel()}) -> {ok, dict()};
          (decompress, Dict :: binary(), #{}) -> {ok, dict()}.

Create a compression or decompression dictionary.

A compression dictionary can be used as a compress_parameters/0 to use a dictionary for compression. Dictionaries allow good compression ratios even for small amounts of data.

A decompression dictionary can be used as a decompress_parameters/0 to use a dictionary for decompression. The same dictionary has to be used for compression as decompression. To verify that the same dictionary is used you can use get_dict_id/1 on the dictionary and compressed data, or just try to decompress as decompression will raise and exception if an incorrect dictionary is given.

The compressionLevel set on a dictionary will override the compressionLevel set in the context/0.

Example:

1> {ok, CDict} = zstd:dict(compress, Dict).
2> Data = lists:duplicate(100, 1).
[1, 1, 1 | _]
3> iolist_size(zstd:compress(Data)).
17
4> iolist_size(zstd:compress(Data, #{ dictionary => CDict, dictIDFlag => false })).
16

As loading a dictionary can be a heavy operations, it is possible to create only a single dict/0 and provide it to multiple context/0.

There is no API exposed in zstd to create a dictionary, instead use the zstd command line tool.

finish(Ctx, Data)

(since OTP 28.0)

-spec finish(Ctx :: context(), Data :: iodata()) -> Result when Result :: {done, erlang:iovec()}.

Finish compressing/decompressing data.

This flushes all output buffers and resets the context/0 so that it can be used for compressing/decompressing again.

Example:

1> {ok, DCtx} = zstd:context(decompress).
2> {continue, D1} = zstd:stream(DCtx, <<40,181,47,253,32>>).
3> {done, D2} = zstd:finish(DCtx, <<2,17,0,0,97,98>>).
4> iolist_to_binary([D1,D2]).
<<"ab">>

get_dict_id(DictOrFrame)

(since OTP 28.0)

-spec get_dict_id(DictOrFrame :: dict() | binary()) -> non_neg_integer().

Get the dictionary ID of a dictionary or a frame.

The dictionary ID 0 represents no dictionary.

Example:

1> {ok, CDict} = zstd:dict(compress, Dict).
2> zstd:get_dict_id(CDict).
1850243626
3> zstd:get_dict_id(zstd:compress("abc")).
0

get_frame_header(Frame)

(since OTP 28.0)

-spec get_frame_header(Frame :: iodata()) ->
                          {ok,
                           #{blockSizeMax => non_neg_integer(),
                             checksumFlag => boolean(),
                             dictID => non_neg_integer(),
                             frameContentSize => non_neg_integer(),
                             frameType => 'ZSTD_frame' | 'ZSTD_skippableFrame',
                             headerSize => non_neg_integer(),
                             windowSize => non_neg_integer()}} |
                          {error, unicode:chardata()}.

Get header of a Zstandard compressed frame.

A compressed Zstandard stream can consist of multiple frames. This function will read metadata from the first frame. This information can be useful when debugging corrupted Zstandard streams.

Example:

1> Compressed = zstd:compress(~"abc").
2> zstd:get_frame_header(Compressed).
{ok,#{frameContentSize => 3,windowSize => 3,blockSizeMax => 3,
      frameType => 'ZSTD_frame',headerSize => 6,
      dictID => 0, checksumFlag => false}}

get_parameter(Ctx, Key)

(since OTP 28.0)

-spec get_parameter(Ctx :: context(), Key :: term()) -> Value :: term().

Get a parameter from a context/0.

See compress_parameters/0 and decompress_parameters/0 for details on which parameters are available and what each parameter does.

Note that it is not possible to get the dictionary and pledgedSrcSize parameters using this API. Instead you can use get_dict_id/1 on the context/0 to get the id of the dictionary used. There is no way to get the pledgedSrcSize.

Returns ok on success, raises an error on failure.

Example:

1> {ok, CCtx} = zstd:context(compress).
{ok, _}
2> zstd:get_parameter(CCtx, compressionLevel).
3
3> zstd:set_parameter(CCtx, compressionLevel, 15).
ok
4> zstd:get_parameter(CCtx, compressionLevel).
15

reset(Ctx)

(since OTP 28.0)

-spec reset(Ctx :: context()) -> ok.

Reset a context while streaming data, returning it to its original state but keeping all parameters set.

By resetting the state, the context can be re-used for other operations even if it is in the middle of a (de)compression stream.

Example:

1> {ok, CCtx} = zstd:context(compress).
2> zstd:stream(CCtx, "a").
{continue, _}
3> zstd:reset(CCtx).
ok
4> {done, Compressed} = zstd:finish(CCtx, "b").
5> zstd:decompress(Compressed).
[~"b"]

set_parameter(Ctx, Key, Value)

(since OTP 28.0)

-spec set_parameter(Ctx :: context(), Key :: term(), Value :: term()) -> ok.

Set a parameter on a context/0.

See compress_parameters/0 and decompress_parameters/0 for details on which parameters are available and what each parameter does.

Returns ok on success, raises an error on failure.

Example:

1> {ok, CCtx} = zstd:context(compress).
{ok, _}
2> ok = zstd:set_parameter(CCtx, compressionLevel, 15).
ok
3> zstd:stream(CCtx, "abc").
{continue, _}
4> catch zstd:set_parameter(CCtx, dictionary, "abc").
{'EXIT', {{zstd_error, <<"Operation not authorized at current processing stage">>}, _}}

stream(Ctx, Data)

(since OTP 28.0)

-spec stream(Ctx :: context(), Data :: iodata()) -> Result
                when
                    Result ::
                        {continue, Remainder :: erlang:iovec(), Output :: binary()} |
                        {continue, Output :: binary()}.

Compress or decompress a stream of data. The last stream of data should be called with finish/2 to complete the compression/decompression.

Example:

1> {ok, CCtx} = zstd:context(compress).
2> {continue, C1} = zstd:stream(CCtx, ~"a").
3> {done, C2} = zstd:finish(CCtx, ~"b").
4> Compressed = iolist_to_binary([C1, C2]).
<<40,181,47,253,0,88,17,0,0,97,98>>
5> zstd:decompress(Compressed).
[<<"ab">>]

beam_lib

This module provides an interface to files created by the BEAM Compiler ("BEAM files").

The format used, a variant of "EA IFF 1985" Standard for Interchange Format Files, divides data into chunks.

Chunk data can be returned as binaries or as compound terms. Compound terms are returned when chunks are referenced by names (atoms) rather than identifiers (strings). The recognized names and the corresponding identifiers are as follows:

atoms ("Atom")

attributes ("Attr")

compile_info ("CInf")

debug_info ("Dbgi")

exports ("ExpT")

imports ("ImpT")

indexed_imports ("ImpT")

labeled_exports ("ExpT")

labeled_locals ("LocT")

literals ("LitT")

locals ("LocT")

documentation ("Docs")

Debug Information/Abstract Code

Option debug_info can be specified to the Compiler (see compile) to have debug information, such as Erlang Abstract Format, stored in the debug_info chunk. Tools such as Debugger and Xref require the debug information to be included.

Warning

Source code can be reconstructed from the debug information. To prevent this, use encrypted debug information (see below).

The debug information can also be removed from BEAM files using strip/1, strip_files/1, and/or strip_release/1.

Reconstruct Source Code

The following example shows how to reconstruct Erlang source code from the debug information in a BEAM file Beam:

{ok,{_,[{abstract_code,{_,AC}}]}} = beam_lib:chunks(Beam,[abstract_code]).
io:fwrite("~s~n", [erl_prettypr:format(erl_syntax:form_list(AC))]).

Encrypted Debug Information

The debug information can be encrypted to keep the source code secret, but still be able to use tools such as Debugger or Xref.

To use encrypted debug information, a key must be provided to the compiler and beam_lib. The key is specified as a string. It is recommended that the string contains at least 32 characters and that both upper and lower case letters as well as digits and special characters are used.

The default type (and currently the only type) of crypto algorithm is des3_cbc, three rounds of DES. The key string is scrambled using erlang:md5/1 to generate the keys used for des3_cbc.

Note

As far as we know by the time of writing, it is infeasible to break des3_cbc encryption without any knowledge of the key. Therefore, as long as the key is kept safe and is unguessable, the encrypted debug information should be safe from intruders.

The key can be provided in the following two ways:

Use Compiler option {debug_info_key,Key}, see compile and function crypto_key_fun/1 to register a fun that returns the key whenever beam_lib must decrypt the debug information. If no such fun is registered, beam_lib instead searches for an .erlang.crypt file, see the next section.

Store the key in a text file named .erlang.crypt. In this case, Compiler option encrypt_debug_info can be used, see compile.

.erlang.crypt

beam_lib searches for .erlang.crypt in the current directory, then the user's home directory and then filename:basedir(user_config, "erlang"). If the file is found and contains a key, beam_lib implicitly creates a crypto key fun and registers it.

File .erlang.crypt is to contain a single list of tuples:

{debug_info, Mode, Module, Key}

Mode is the type of crypto algorithm; currently, the only allowed value is des3_cbc. Module is either an atom, in which case Key is only used for the module Module, or [], in which case Key is used for all modules. Key is the non-empty key string.

Key in the first tuple where both Mode and Module match is used.

The following is an example of an .erlang.crypt file that returns the same key for all modules:

[{debug_info, des3_cbc, [], "%>7}|pc/DM6Cga*68$Mw]L#&_Gejr]G^"}].

The following is a slightly more complicated example of an .erlang.crypt providing one key for module t and another key for all other modules:

[{debug_info, des3_cbc, t, "My KEY"},
 {debug_info, des3_cbc, [], "%>7}|pc/DM6Cga*68$Mw]L#&_Gejr]G^"}].

Note

Do not use any of the keys in these examples. Use your own keys.

Summary

Types

abst_code()

It is not checked that the forms conform to the abstract format indicated by AbstVersion. no_abstract_code means that chunk "Abst" is present, but empty.

attrib_entry()

beam()

Each of the functions described below accept either the filename (as a string) or a binary containing the BEAM module.

chnk_rsn()

chunkdata()

The list of attributes is sorted on Attribute (in attrib_entry/0) and each attribute name occurs once in the list. The attribute values occur in the same order as in the file. The lists of functions are also sorted.

chunkid()

"Attr" | "CInf" | "Dbgi" | "ExpT" | "ImpT" | "LocT" | "AtU8" | "Docs"

chunkname()

chunkref()

cmp_rsn()

compinfo_entry()

crypto_fun()

crypto_fun_arg()

dataB()

debug_info()

The format stored in the debug_info chunk.

docs()

EEP-48 documentation format

forms()

index()

info_rsn()

label()

labeled_entry()

literals()

mode()

Functions

all_chunks(File)

Reads chunk data for all chunks.

build_module(Chunks)

Builds a BEAM module (as a binary) from a list of chunks.

chunks(Beam, ChunkRefs)

Reads chunk data for selected chunks references. The order of the returned list of chunk data is determined by the order of the list of chunks references.

chunks(Beam, ChunkRefs, Options)

Reads chunk data for selected chunks references. The order of the returned list of chunk data is determined by the order of the list of chunks references.

clear_crypto_key_fun()

Unregisters the crypto key fun and terminates the process holding it, started by crypto_key_fun/1.

cmp(Beam1, Beam2)

Compares the contents of two BEAM files.

cmp_dirs(Dir1, Dir2)

Compares the BEAM files in two directories.

crypto_key_fun(CryptoKeyFun)

Registers an unary fun that is called if beam_lib must read an debug_info chunk that has been encrypted. The fun is held in a process that is started by the function.

diff_dirs(Dir1, Dir2)

Compares the BEAM files in two directories as cmp_dirs/2, but the names of files that exist in only one directory or are different are presented on standard output.

format_error(Reason)

For a specified error returned by any function in this module, this function returns a descriptive string of the error in English. For file errors, function file:format_error(Posix) is to be called.

info(Beam)

Returns a list containing some information about a BEAM file as tuples {Item, Info}

md5(Beam)

Calculates an MD5 redundancy check for the code of the module (compilation date and other attributes are not included).

strip(Beam1)

Removes all chunks from a BEAM file except those used by the loader.

strip(Beam1, AdditionalChunks)

Removes all chunks from a BEAM file except those used by the loader or mentioned in AdditionalChunks.

strip_files(Files)

Removes all chunks except those used by the loader from Files.

strip_files(Files, AdditionalChunks)

Removes all chunks except those used by the loader or mentioned in AdditionalChunks from Files.

strip_release(Dir)

Removes all chunks except those used by the loader from the BEAM files of a release.

strip_release(Dir, AdditionalChunks)

Removes all chunks except those used by the loader or mentioned in AdditionalChunks.

version(Beam)

Returns the module version or versions. A version is defined by module attribute -vsn(Vsn).

Types

abst_code()

(not exported)

-type abst_code() :: {AbstVersion :: atom(), forms()} | no_abstract_code.

It is not checked that the forms conform to the abstract format indicated by AbstVersion. no_abstract_code means that chunk "Abst" is present, but empty.

For modules compiled with OTP 20 onwards, the abst_code chunk is automatically computed from the debug_info chunk.

attrib_entry()

-type attrib_entry() :: {Attribute :: atom(), [AttributeValue :: term()]}.

beam()

-type beam() :: file:filename() | binary().

Each of the functions described below accept either the filename (as a string) or a binary containing the BEAM module.

chnk_rsn()

-type chnk_rsn() ::
          {unknown_chunk, file:filename(), atom()} |
          {key_missing_or_invalid, file:filename(), abstract_code | debug_info} |
          {missing_backend, file:filename(), module()} |
          info_rsn().

chunkdata()

(not exported)

-type chunkdata() ::
          {chunkid(), dataB()} |
          {abstract_code, abst_code()} |
          {debug_info, debug_info()} |
          {attributes, [attrib_entry()]} |
          {compile_info, [compinfo_entry()]} |
          {exports, [{atom(), arity()}]} |
          {labeled_exports, [labeled_entry()]} |
          {imports, [mfa()]} |
          {indexed_imports, [{index(), module(), Function :: atom(), arity()}]} |
          {locals, [{atom(), arity()}]} |
          {labeled_locals, [labeled_entry()]} |
          {atoms, [{integer(), atom()}]} |
          {documentation, docs()} |
          {literals, literals()}.

The list of attributes is sorted on Attribute (in attrib_entry/0) and each attribute name occurs once in the list. The attribute values occur in the same order as in the file. The lists of functions are also sorted.

chunkid()

-type chunkid() :: nonempty_string().

"Attr" | "CInf" | "Dbgi" | "ExpT" | "ImpT" | "LocT" | "AtU8" | "Docs"

chunkname()

(not exported)

-type chunkname() ::
          abstract_code | debug_info | attributes | compile_info | exports | labeled_exports | imports |
          indexed_imports | locals | labeled_locals | atoms | documentation | literals.

chunkref()

(not exported)

-type chunkref() :: chunkname() | chunkid().

cmp_rsn()

(not exported)

-type cmp_rsn() ::
          {modules_different, module(), module()} |
          {chunks_different, chunkid()} |
          different_chunks |
          info_rsn().

compinfo_entry()

-type compinfo_entry() :: {InfoKey :: atom(), term()}.

crypto_fun()

(not exported)

-type crypto_fun() :: fun((crypto_fun_arg()) -> term()).

crypto_fun_arg()

(not exported)

-type crypto_fun_arg() :: init | clear | {debug_info, mode(), module(), file:filename()}.

dataB()

(not exported)

-type dataB() :: binary().

debug_info()

(not exported)

-type debug_info() :: {DbgiVersion :: atom(), Backend :: module(), Data :: term()} | no_debug_info.

The format stored in the debug_info chunk.

To retrieve particular code representation from the backend, Backend:debug_info(Format, Module, Data, Opts) must be invoked. Format is an atom, such as erlang_v1 for the Erlang Abstract Format or core_v1 for Core Erlang. Module is the module represented by the beam file and Data is the value stored in the debug info chunk. Opts is any list of values supported by the Backend. Backend:debug_info/4 must return {ok, Code} or {error, Term}.

Developers must always invoke the debug_info/4 function and never rely on the Data stored in the debug_info chunk, as it is opaque and may change at any moment. no_debug_info means that chunk "Dbgi" is present, but empty.

docs()

(not exported)

-type docs() ::
          #docs_v1{anno :: term(),
                   beam_language :: term(),
                   format :: term(),
                   module_doc :: term(),
                   metadata :: term(),
                   docs :: term()}.

EEP-48 documentation format

forms()

(not exported)

-type forms() :: [erl_parse:abstract_form() | erl_parse:form_info()].

index()

(not exported)

-type index() :: non_neg_integer().

info_rsn()

(not exported)

-type info_rsn() ::
          {chunk_too_big,
           file:filename(),
           chunkid(),
           ChunkSize :: non_neg_integer(),
           FileSize :: non_neg_integer()} |
          {invalid_beam_file, file:filename(), Position :: non_neg_integer()} |
          {invalid_chunk, file:filename(), chunkid()} |
          {missing_chunk, file:filename(), chunkid()} |
          {not_a_beam_file, file:filename()} |
          {file_error, file:filename(), file:posix()}.

label()

-type label() :: integer().

labeled_entry()

-type labeled_entry() :: {Function :: atom(), arity(), label()}.

literals()

(not exported)

-type literals() :: {index(), term()}.

mode()

(not exported)

-type mode() :: des3_cbc.

Functions

all_chunks(File)

(since OTP 18.2)

-spec all_chunks(beam()) -> {ok, module(), [{chunkid(), dataB()}]} | {error, beam_lib, info_rsn()}.

Reads chunk data for all chunks.

build_module(Chunks)

(since OTP 18.2)

-spec build_module(Chunks) -> {ok, Binary} when Chunks :: [{chunkid(), dataB()}], Binary :: binary().

Builds a BEAM module (as a binary) from a list of chunks.

chunks(Beam, ChunkRefs)

-spec chunks(Beam, ChunkRefs) -> {ok, {module(), [chunkdata()]}} | {error, beam_lib, chnk_rsn()}
                when Beam :: beam(), ChunkRefs :: [chunkref()].

Reads chunk data for selected chunks references. The order of the returned list of chunk data is determined by the order of the list of chunks references.

chunks(Beam, ChunkRefs, Options)

-spec chunks(Beam, ChunkRefs, Options) ->
                {ok, {module(), [ChunkResult]}} | {error, beam_lib, chnk_rsn()}
                when
                    Beam :: beam(),
                    ChunkRefs :: [chunkref()],
                    Options :: [allow_missing_chunks],
                    ChunkResult :: chunkdata() | {ChunkRef :: chunkref(), missing_chunk}.

Reads chunk data for selected chunks references. The order of the returned list of chunk data is determined by the order of the list of chunks references.

By default, if any requested chunk is missing in Beam, an error tuple is returned. However, if option allow_missing_chunks is specified, a result is returned even if chunks are missing. In the result list, any missing chunks are represented as {ChunkRef,missing_chunk}. Notice however that if chunk "Atom" is missing, that is considered a fatal error and the return value is an error tuple.

clear_crypto_key_fun()

-spec clear_crypto_key_fun() -> undefined | {ok, Result} when Result :: undefined | term().

Unregisters the crypto key fun and terminates the process holding it, started by crypto_key_fun/1.

Returns either {ok, undefined} if no crypto key fun is registered, or {ok, Term}, where Term is the return value from CryptoKeyFun(clear), see crypto_key_fun/1.

cmp(Beam1, Beam2)

-spec cmp(Beam1, Beam2) -> ok | {error, beam_lib, cmp_rsn()} when Beam1 :: beam(), Beam2 :: beam().

Compares the contents of two BEAM files.

If the module names are the same, and all chunks except for chunk "CInf" (the chunk containing the compilation information that is returned by Module:module_info(compile)) have the same contents in both files, ok is returned. Otherwise an error message is returned.

cmp_dirs(Dir1, Dir2)

-spec cmp_dirs(Dir1, Dir2) -> {Only1, Only2, Different} | {error, beam_lib, Reason}
                  when
                      Dir1 :: atom() | file:filename(),
                      Dir2 :: atom() | file:filename(),
                      Only1 :: [file:filename()],
                      Only2 :: [file:filename()],
                      Different :: [{Filename1 :: file:filename(), Filename2 :: file:filename()}],
                      Reason :: {not_a_directory, term()} | info_rsn().

Compares the BEAM files in two directories.

Only files with extension ".beam" are compared. BEAM files that exist only in directory Dir1 (Dir2) are returned in Only1 (Only2). BEAM files that exist in both directories but are considered different by cmp/2 are returned as pairs {Filename1, Filename2}, where Filename1 (Filename2) exists in directory Dir1 (Dir2).

crypto_key_fun(CryptoKeyFun)

-spec crypto_key_fun(CryptoKeyFun) -> ok | {error, Reason}
                        when CryptoKeyFun :: crypto_fun(), Reason :: badfun | exists | term().

Registers an unary fun that is called if beam_lib must read an debug_info chunk that has been encrypted. The fun is held in a process that is started by the function.

If a fun is already registered when attempting to register a fun, {error, exists} is returned.

The fun must handle the following arguments:

CryptoKeyFun(init) -> ok | {ok, NewCryptoKeyFun} | {error, Term}

Called when the fun is registered, in the process that holds the fun. Here the crypto key fun can do any necessary initializations. If {ok, NewCryptoKeyFun} is returned, NewCryptoKeyFun is registered instead of CryptoKeyFun. If {error, Term} is returned, the registration is aborted and crypto_key_fun/1 also returns {error, Term}.

CryptoKeyFun({debug_info, Mode, Module, Filename}) -> Key

Called when the key is needed for module Module in the file named Filename. Mode is the type of crypto algorithm; currently, the only possible value is des3_cbc. The call is to fail (raise an exception) if no key is available.

CryptoKeyFun(clear) -> term()

Called before the fun is unregistered. Here any cleaning up can be done. The return value is not important, but is passed back to the caller of clear_crypto_key_fun/0 as part of its return value.

diff_dirs(Dir1, Dir2)

-spec diff_dirs(Dir1, Dir2) -> ok | {error, beam_lib, Reason}
                   when
                       Dir1 :: atom() | file:filename(),
                       Dir2 :: atom() | file:filename(),
                       Reason :: {not_a_directory, term()} | info_rsn().

Compares the BEAM files in two directories as cmp_dirs/2, but the names of files that exist in only one directory or are different are presented on standard output.

format_error(Reason)

-spec format_error(Reason) -> io_lib:chars() when Reason :: term().

For a specified error returned by any function in this module, this function returns a descriptive string of the error in English. For file errors, function file:format_error(Posix) is to be called.

info(Beam)

-spec info(Beam) -> [InfoPair] | {error, beam_lib, info_rsn()}
              when
                  Beam :: beam(),
                  InfoPair ::
                      {file, Filename :: file:filename()} |
                      {binary, Binary :: binary()} |
                      {module, Module :: module()} |
                      {chunks,
                       [{ChunkId :: chunkid(), Pos :: non_neg_integer(), Size :: non_neg_integer()}]}.

Returns a list containing some information about a BEAM file as tuples {Item, Info}:

{file, Filename} | {binary, Binary} - The name (string) of the BEAM file, or the binary from which the information was extracted.

{module, Module} - The name (atom) of the module.

{chunks, [{ChunkId, Pos, Size}]} - For each chunk, the identifier (string) and the position and size of the chunk data, in bytes.

md5(Beam)

-spec md5(Beam) -> {ok, {module(), MD5}} | {error, beam_lib, chnk_rsn()}
             when Beam :: beam(), MD5 :: binary().

Calculates an MD5 redundancy check for the code of the module (compilation date and other attributes are not included).

strip(Beam1)

-spec strip(Beam1) -> {ok, {module(), Beam2}} | {error, beam_lib, info_rsn()}
               when Beam1 :: beam(), Beam2 :: beam().

Removes all chunks from a BEAM file except those used by the loader.

In particular, the debug information (chunk debug_info and abstract_code) is removed.

strip(Beam1, AdditionalChunks)

(since OTP 22.0)

-spec strip(Beam1, AdditionalChunks) -> {ok, {module(), Beam2}} | {error, beam_lib, info_rsn()}
               when Beam1 :: beam(), AdditionalChunks :: [chunkid()], Beam2 :: beam().

Removes all chunks from a BEAM file except those used by the loader or mentioned in AdditionalChunks.

In particular, the debug information (chunk debug_info and abstract_code) is removed.

strip_files(Files)

-spec strip_files(Files) -> {ok, [{module(), Beam}]} | {error, beam_lib, info_rsn()}
                     when Files :: [beam()], Beam :: beam().

Removes all chunks except those used by the loader from Files.

In particular, the debug information (chunk debug_info and abstract_code) is removed. The returned list contains one element for each specified filename, in the same order as in Files.

strip_files(Files, AdditionalChunks)

(since OTP 22.0)

-spec strip_files(Files, AdditionalChunks) -> {ok, [{module(), Beam}]} | {error, beam_lib, info_rsn()}
                     when Files :: [beam()], AdditionalChunks :: [chunkid()], Beam :: beam().

Removes all chunks except those used by the loader or mentioned in AdditionalChunks from Files.

In particular, the debug information (chunk debug_info and abstract_code) is removed. The returned list contains one element for each specified filename, in the same order as in Files.

strip_release(Dir)

-spec strip_release(Dir) -> {ok, [{module(), file:filename()}]} | {error, beam_lib, Reason}
                       when
                           Dir :: atom() | file:filename(),
                           Reason :: {not_a_directory, term()} | info_rsn().

Removes all chunks except those used by the loader from the BEAM files of a release.

Dir is to be the installation root directory. For example, the current OTP release can be stripped with the call beam_lib:strip_release(code:root_dir()).

strip_release(Dir, AdditionalChunks)

(since OTP 22.0)

-spec strip_release(Dir, AdditionalChunks) ->
                       {ok, [{module(), file:filename()}]} | {error, beam_lib, Reason}
                       when
                           Dir :: atom() | file:filename(),
                           AdditionalChunks :: [chunkid()],
                           Reason :: {not_a_directory, term()} | info_rsn().

Removes all chunks except those used by the loader or mentioned in AdditionalChunks.

Dir is to be the installation root directory. For example, the current OTP release can be stripped with the call beam_lib:strip_release(code:root_dir(),[documentation]).

version(Beam)

-spec version(Beam) -> {ok, {module(), [Version :: term()]}} | {error, beam_lib, chnk_rsn()}
                 when Beam :: beam().

Returns the module version or versions. A version is defined by module attribute -vsn(Vsn).

If this attribute is not specified, the version defaults to the checksum of the module. Notice that if version Vsn is not a list, it is made into one, that is {ok,{Module,[Vsn]}} is returned. If there are many -vsn module attributes, the result is the concatenated list of versions.

Examples:

1> beam_lib:version(a). % -vsn(1).
{ok,{a,[1]}}
2> beam_lib:version(b). % -vsn([1]).
{ok,{b,[1]}}
3> beam_lib:version(c). % -vsn([1]). -vsn(2).
{ok,{c,[1,2]}}
4> beam_lib:version(d). % no -vsn attribute
{ok,{d,[275613208176997377698094100858909383631]}}

epp

An Erlang code preprocessor.

The Erlang code preprocessor includes functions that are used by the compile module to preprocess macros and include files before the parsing takes place.

The Erlang source file encoding is selected by a comment in one of the first two lines of the source file. The first string matching the regular expression coding\s*[:=]\s*([-a-zA-Z0-9])+ selects the encoding. If the matching string is not a valid encoding, it is ignored. The valid encodings are Latin-1 and UTF-8, where the case of the characters can be chosen freely.

Examples:

%% coding: utf-8

%% For this file we have chosen encoding = Latin-1

%% -*- coding: latin-1 -*-

Error Information

ErrorInfo is the standard ErrorInfo structure that is returned from all I/O modules. The format is as follows:

{ErrorLine, Module, ErrorDescriptor}

A string describing the error is obtained with the following call:

Module:format_error(ErrorDescriptor)

See Also

erl_parse

Summary

Types

epp_handle()

Handle to the epp server.

macros()

source_encoding()

warning_info()

Functions

close(Epp)

Closes the preprocessing of a file.

default_encoding()

Returns the default encoding of Erlang source files.

encoding_to_string(Encoding)

Returns a string representation of an encoding. The string is recognized by read_encoding/1,2, read_encoding_from_binary/1,2, and set_encoding/1,2 as a valid encoding.

format_error(ErrorDescriptor)

Takes an ErrorDescriptor and returns a string that describes the error or warning. This function is usually called implicitly when processing an ErrorInfo structure (see section Error Information).

open(Options)

Opens a file for preprocessing.

open(FileName, IncludePath)

Equivalent to epp:open([{name, FileName}, {includes, IncludePath}]).

open(FileName, IncludePath, PredefMacros)

Equivalent to epp:open([{name, FileName}, {includes, IncludePath}, {macros, PredefMacros}]).

parse_erl_form(Epp)

Returns the next Erlang form from the opened Erlang source file. Tuple {eof, Location} is returned at the end of the file. The first form corresponds to an implicit attribute -file(File,1)., where File is the file name.

parse_file(FileName, Options)

Preprocesses and parses an Erlang source file. Notice that tuple {eof, Location} returned at the end of the file is included as a "form".

parse_file(FileName, IncludePath, PredefMacros)

Equivalent to epp:parse_file(FileName, [{includes, IncludePath}, {macros, PredefMacros}]).

read_encoding(FileName)

Equivalent to read_encoding/2.

read_encoding(FileName, Options)

Read the encoding from a file. Returns the read encoding, or none if no valid encoding is found.

read_encoding_from_binary(Binary)

Equivalent to read_encoding_from_binary/2.

read_encoding_from_binary(Binary, Options)

Read the encoding from a binary. Returns the read encoding, or none if no valid encoding is found.

scan_erl_form(Epp)

Returns the raw tokens of the next Erlang form from the opened Erlang source file. A tuple {eof, Line} is returned at the end of the file. The first form corresponds to an implicit attribute -file(File,1)., where File is the file name.

scan_file(FileName, Options)

Preprocesses an Erlang source file returning a list of the lists of raw tokens of each form. Notice that the tuple {eof, Line} returned at the end of the file is included as a "form", and any failures to scan a form are included in the list as tuples {error, ErrorInfo}.

set_encoding(File)

Reads the encoding from an I/O device and sets the encoding of the device accordingly. The position of the I/O device referenced by File is not affected. If no valid encoding can be read from the I/O device, the encoding of the I/O device is set to the default encoding.

set_encoding(File, Default)

Reads the encoding from an I/O device and sets the encoding of the device accordingly. The position of the I/O device referenced by File is not affected. If no valid encoding can be read from the I/O device, the encoding of the I/O device is set to the encoding specified by Default.

Types

epp_handle()

(not exported)

-type epp_handle() :: pid().

Handle to the epp server.

macros()

(not exported)

-type macros() :: [atom() | {atom(), term()} | {atom(), term(), redefine}].

source_encoding()

-type source_encoding() :: latin1 | utf8.

warning_info()

(not exported)

-type warning_info() :: {erl_anno:location(), module(), term()}.

Functions

close(Epp)

-spec close(Epp) -> ok when Epp :: epp_handle().

Closes the preprocessing of a file.

default_encoding()

(since OTP R16B)

-spec default_encoding() -> source_encoding().

Returns the default encoding of Erlang source files.

encoding_to_string(Encoding)

(since OTP R16B)

-spec encoding_to_string(Encoding) -> string() when Encoding :: source_encoding().

Returns a string representation of an encoding. The string is recognized by read_encoding/1,2, read_encoding_from_binary/1,2, and set_encoding/1,2 as a valid encoding.

format_error(ErrorDescriptor)

(since OTP R14B03)

-spec format_error(ErrorDescriptor) -> io_lib:chars() when ErrorDescriptor :: term().

Takes an ErrorDescriptor and returns a string that describes the error or warning. This function is usually called implicitly when processing an ErrorInfo structure (see section Error Information).

open(Options)

(since OTP 17.0)

-spec open(Options) -> {ok, Epp} | {ok, Epp, Extra} | {error, ErrorDescriptor}
              when
                  Options ::
                      [{default_encoding, DefEncoding :: source_encoding()} |
                       {includes, IncludePath :: [DirectoryName :: file:name()]} |
                       {source_name, SourceName :: file:name()} |
                       {deterministic, Enabled :: boolean()} |
                       {macros, PredefMacros :: macros()} |
                       {name, FileName :: file:name()} |
                       {location, StartLocation :: erl_anno:location()} |
                       {fd, FileDescriptor :: file:io_device()} |
                       extra |
                       {compiler_internal, [term()]}],
                  Epp :: epp_handle(),
                  Extra :: [{encoding, source_encoding() | none}],
                  ErrorDescriptor :: term().

Opens a file for preprocessing.

If you want to change the file name of the implicit -file() attributes inserted during preprocessing, you can do with {source_name, SourceName}. If unset it will default to the name of the opened file.

Setting {deterministic, Enabled} will additionally reduce the file name of the implicit -file() attributes inserted during preprocessing to only the basename of the path.

If extra is specified in Options, the return value is {ok, Epp, Extra} instead of {ok, Epp}.

The option location is forwarded to the Erlang token scanner, see erl_scan:tokens/3,4.

The {compiler_internal,term()} option is forwarded to the Erlang token scanner, see {compiler_internal,term()}.

open(FileName, IncludePath)

-spec open(FileName, IncludePath) -> {ok, Epp} | {error, ErrorDescriptor}
              when
                  FileName :: file:name(),
                  IncludePath :: [DirectoryName :: file:name()],
                  Epp :: epp_handle(),
                  ErrorDescriptor :: term().

Equivalent to epp:open([{name, FileName}, {includes, IncludePath}]).

open(FileName, IncludePath, PredefMacros)

-spec open(FileName, IncludePath, PredefMacros) -> {ok, Epp} | {error, ErrorDescriptor}
              when
                  FileName :: file:name(),
                  IncludePath :: [DirectoryName :: file:name()],
                  PredefMacros :: macros(),
                  Epp :: epp_handle(),
                  ErrorDescriptor :: term().

Equivalent to epp:open([{name, FileName}, {includes, IncludePath}, {macros, PredefMacros}]).

parse_erl_form(Epp)

-spec parse_erl_form(Epp) ->
                        {ok, AbsForm} | {error, ErrorInfo} | {warning, WarningInfo} | {eof, Location}
                        when
                            Epp :: epp_handle(),
                            AbsForm :: erl_parse:abstract_form(),
                            Location :: erl_anno:location(),
                            ErrorInfo :: erl_scan:error_info() | erl_parse:error_info(),
                            WarningInfo :: warning_info().

Returns the next Erlang form from the opened Erlang source file. Tuple {eof, Location} is returned at the end of the file. The first form corresponds to an implicit attribute -file(File,1)., where File is the file name.

parse_file(FileName, Options)

(since OTP 17.0)

-spec parse_file(FileName, Options) -> {ok, [Form]} | {ok, [Form], Extra} | {error, OpenError}
                    when
                        FileName :: file:name(),
                        Options ::
                            [{includes, IncludePath :: [DirectoryName :: file:name()]} |
                             {source_name, SourceName :: file:name()} |
                             {macros, PredefMacros :: macros()} |
                             {default_encoding, DefEncoding :: source_encoding()} |
                             {location, StartLocation :: erl_anno:location()} |
                             {reserved_word_fun, Fun :: fun((atom()) -> boolean())} |
                             {features, [Feature :: atom()]} |
                             {deterministic, boolean()} |
                             extra |
                             {compiler_internal, [term()]}],
                        Form :: erl_parse:abstract_form() | {error, ErrorInfo} | {eof, Location},
                        Location :: erl_anno:location(),
                        ErrorInfo :: erl_scan:error_info() | erl_parse:error_info(),
                        Extra :: [{encoding, source_encoding() | none}],
                        OpenError :: file:posix() | badarg | system_limit.

Preprocesses and parses an Erlang source file. Notice that tuple {eof, Location} returned at the end of the file is included as a "form".

If you want to change the file name of the implicit -file() attributes inserted during preprocessing, you can do with {source_name, SourceName}. If unset it will default to the name of the opened file.

If extra is specified in Options, the return value is {ok, [Form], Extra} instead of {ok, [Form]}.

The option location is forwarded to the Erlang token scanner, see erl_scan:tokens/3,4.

The {compiler_internal,term()} option is forwarded to the Erlang token scanner, see {compiler_internal,term()}.

parse_file(FileName, IncludePath, PredefMacros)

-spec parse_file(FileName, IncludePath, PredefMacros) -> {ok, [Form]} | {error, OpenError}
                    when
                        FileName :: file:name(),
                        IncludePath :: [DirectoryName :: file:name()],
                        Form :: erl_parse:abstract_form() | {error, ErrorInfo} | {eof, Location},
                        PredefMacros :: macros(),
                        Location :: erl_anno:location(),
                        ErrorInfo :: erl_scan:error_info() | erl_parse:error_info(),
                        OpenError :: file:posix() | badarg | system_limit.

Equivalent to epp:parse_file(FileName, [{includes, IncludePath}, {macros, PredefMacros}]).

read_encoding(FileName)

(since OTP R16B)

-spec read_encoding(FileName) -> source_encoding() | none when FileName :: file:name().

Equivalent to read_encoding/2.

read_encoding(FileName, Options)

(since OTP R16B)

-spec read_encoding(FileName, Options) -> source_encoding() | none
                       when
                           FileName :: file:name(),
                           Options :: [Option],
                           Option :: {in_comment_only, boolean()}.

Read the encoding from a file. Returns the read encoding, or none if no valid encoding is found.

Option in_comment_only is true by default, which is correct for Erlang source files. If set to false, the encoding string does not necessarily have to occur in a comment.

read_encoding_from_binary(Binary)

(since OTP R16B)

-spec read_encoding_from_binary(Binary) -> source_encoding() | none when Binary :: binary().

Equivalent to read_encoding_from_binary/2.

read_encoding_from_binary(Binary, Options)

(since OTP R16B)

-spec read_encoding_from_binary(Binary, Options) -> source_encoding() | none
                                   when
                                       Binary :: binary(),
                                       Options :: [Option],
                                       Option :: {in_comment_only, boolean()}.

Read the encoding from a binary. Returns the read encoding, or none if no valid encoding is found.

Option in_comment_only is true by default, which is correct for Erlang source files. If set to false, the encoding string does not necessarily have to occur in a comment.

scan_erl_form(Epp)

(since OTP R13B03)

-spec scan_erl_form(Epp) -> {ok, Tokens} | {error, ErrorInfo} | {warning, WarningInfo} | {eof, Line}
                       when
                           Epp :: epp_handle(),
                           Tokens :: erl_scan:tokens(),
                           Line :: erl_anno:line(),
                           ErrorInfo :: erl_scan:error_info() | erl_parse:error_info(),
                           WarningInfo :: warning_info().

Returns the raw tokens of the next Erlang form from the opened Erlang source file. A tuple {eof, Line} is returned at the end of the file. The first form corresponds to an implicit attribute -file(File,1)., where File is the file name.

scan_file(FileName, Options)

(since OTP 24.0)

-spec scan_file(FileName, Options) -> {ok, [Form], Extra} | {error, OpenError}
                   when
                       FileName :: file:name(),
                       Options ::
                           [{includes, IncludePath :: [DirectoryName :: file:name()]} |
                            {source_name, SourceName :: file:name()} |
                            {macros, PredefMacros :: macros()} |
                            {default_encoding, DefEncoding :: source_encoding()}],
                       Form :: erl_scan:tokens() | {error, ErrorInfo} | {eof, Loc},
                       Loc :: erl_anno:location(),
                       ErrorInfo :: erl_scan:error_info(),
                       Extra :: [{encoding, source_encoding() | none}],
                       OpenError :: file:posix() | badarg | system_limit.

Preprocesses an Erlang source file returning a list of the lists of raw tokens of each form. Notice that the tuple {eof, Line} returned at the end of the file is included as a "form", and any failures to scan a form are included in the list as tuples {error, ErrorInfo}.

set_encoding(File)

(since OTP R16B)

-spec set_encoding(File) -> source_encoding() | none when File :: io:device().

Reads the encoding from an I/O device and sets the encoding of the device accordingly. The position of the I/O device referenced by File is not affected. If no valid encoding can be read from the I/O device, the encoding of the I/O device is set to the default encoding.

Returns the read encoding, or none if no valid encoding is found.

set_encoding(File, Default)

(since OTP 17.0)

-spec set_encoding(File, Default) -> source_encoding() | none
                      when Default :: source_encoding(), File :: io:device().

Reads the encoding from an I/O device and sets the encoding of the device accordingly. The position of the I/O device referenced by File is not affected. If no valid encoding can be read from the I/O device, the encoding of the I/O device is set to the encoding specified by Default.

Returns the read encoding, or none if no valid encoding is found.

erl_anno

Abstract datatype for the annotations of the Erlang Compiler.

This module provides an abstract type that is used by the Erlang Compiler and its helper modules for holding data such as column, line number, and text. The data type is a collection of annotations as described in the following.

The Erlang Token Scanner returns tokens with a subset of the following annotations, depending on the options:

column - The column where the token begins.

location - The line and column where the token begins, or just the line if the column is unknown.

text - The token's text.

From this, the following annotation is derived:

line - The line where the token begins.

This module also supports the following annotations, which are used by various modules:

file - A filename.

generated - A Boolean indicating if the abstract code is compiler-generated. The Erlang Compiler does not emit warnings for such code.

record - A Boolean indicating if the origin of the abstract code is a record. Used by Dialyzer to assign types to tuple elements.

The functions column(), end_location(), line(), location(), and text() in the erl_scan module can be used for inspecting annotations in tokens.

The functions anno_from_term(), anno_to_term(), fold_anno(), map_anno(), mapfold_anno(), and new_anno(), in the erl_parse module can be used for manipulating annotations in abstract code.

See Also

erl_parse, erl_scan

Summary

Types

anno()

A collection of annotations.

anno_term()

The term representing a collection of annotations. It is either a location/0 or a list of key-value pairs.

column()

filename()

generated()

line()

location()

record()

text()

Functions

column(Anno)

Returns the column of the annotations Anno.

end_location(Anno)

Returns the end location of the annotations Anno.

file(Anno)

Returns the filename of the annotations Anno. If there is no filename, undefined is returned.

from_term(Term)

Returns annotations with representation Term.

generated(Anno)

Returns true if annotations Anno is marked as generated. The default is to return false.

is_anno(Term)

Returns true if Term is a collection of annotations, otherwise false.

line(Anno)

Returns the line of the annotations Anno.

location(Anno)

Returns the location of the annotations Anno. If there is no location, a zero line number is returned.

new(Location)

Creates a new collection of annotations given a location.

set_end_location(Location, Anno)

Modifies the end location of the annotations Anno.

set_file(File, Anno)

Modifies the filename of the annotations Anno.

set_generated(Generated, Anno)

Modifies the generated marker of the annotations Anno.

set_line(Line, Anno)

Modifies the line of the annotations Anno.

set_location(Location, Anno)

Modifies the location of the annotations Anno.

set_record(Record, Anno)

Modifies the record marker of the annotations Anno.

set_text(Text, Anno)

Modifies the text of the annotations Anno.

text(Anno)

Returns the text of the annotations Anno. If there is no text, undefined is returned.

to_term(Anno)

Returns the term representing the annotations Anno.

Types

anno()

(since OTP 18.0)

-opaque anno()

A collection of annotations.

anno_term()

(since OTP 18.0)

-type anno_term() :: term().

The term representing a collection of annotations. It is either a location/0 or a list of key-value pairs.

column()

(since OTP 18.0)

-type column() :: pos_integer().

filename()

(not exported) (since OTP 18.0)

-type filename() :: file:filename_all().

generated()

(not exported) (since OTP 18.0)

-type generated() :: boolean().

line()

(since OTP 18.0)

-nominal line() :: non_neg_integer().

location()

(since OTP 18.0)

-nominal location() :: line() | {line(), column()}.

record()

(not exported) (since OTP 18.0)

-type record() :: boolean().

text()

(since OTP 18.0)

-type text() :: string().

Functions

column(Anno)

(since OTP 18.0)

-spec column(Anno) -> column() | undefined when Anno :: anno().

Returns the column of the annotations Anno.

end_location(Anno)

(since OTP 18.0)

-spec end_location(Anno) -> location() | undefined when Anno :: anno().

Returns the end location of the annotations Anno.

If the end location annotation is present, its value is returned. Otherwise, if the text annotation is present, the end location is inferred from the location and the text. Finally, if there is no text, undefined is returned.

file(Anno)

(since OTP 18.0)

-spec file(Anno) -> filename() | undefined when Anno :: anno().

Returns the filename of the annotations Anno. If there is no filename, undefined is returned.

from_term(Term)

(since OTP 18.0)

-spec from_term(Term) -> Anno when Term :: anno_term(), Anno :: anno().

Returns annotations with representation Term.

See also to_term().

generated(Anno)

(since OTP 18.0)

-spec generated(Anno) -> generated() when Anno :: anno().

Returns true if annotations Anno is marked as generated. The default is to return false.

is_anno(Term)

(since OTP 18.0)

-spec is_anno(Term) -> boolean() when Term :: any().

Returns true if Term is a collection of annotations, otherwise false.

line(Anno)

(since OTP 18.0)

-spec line(Anno) -> line() when Anno :: anno().

Returns the line of the annotations Anno.

location(Anno)

(since OTP 18.0)

-spec location(Anno) -> location() when Anno :: anno().

Returns the location of the annotations Anno. If there is no location, a zero line number is returned.

new(Location)

(since OTP 18.0)

-spec new(Location) -> anno() when Location :: location().

Creates a new collection of annotations given a location.

set_end_location(Location, Anno)

(since OTP 28.0)

-spec set_end_location(Location, Anno) -> Anno when Location :: location(), Anno :: anno().

Modifies the end location of the annotations Anno.

set_file(File, Anno)

(since OTP 18.0)

-spec set_file(File, Anno) -> Anno when File :: filename(), Anno :: anno().

Modifies the filename of the annotations Anno.

set_generated(Generated, Anno)

(since OTP 18.0)

-spec set_generated(Generated, Anno) -> Anno when Generated :: generated(), Anno :: anno().

Modifies the generated marker of the annotations Anno.

set_line(Line, Anno)

(since OTP 18.0)

-spec set_line(Line, Anno) -> Anno when Line :: line(), Anno :: anno().

Modifies the line of the annotations Anno.

set_location(Location, Anno)

(since OTP 18.0)

-spec set_location(Location, Anno) -> Anno when Location :: location(), Anno :: anno().

Modifies the location of the annotations Anno.

set_record(Record, Anno)

(since OTP 18.0)

-spec set_record(Record, Anno) -> Anno when Record :: record(), Anno :: anno().

Modifies the record marker of the annotations Anno.

set_text(Text, Anno)

(since OTP 18.0)

-spec set_text(Text, Anno) -> Anno when Text :: text(), Anno :: anno().

Modifies the text of the annotations Anno.

text(Anno)

(since OTP 18.0)

-spec text(Anno) -> text() | undefined when Anno :: anno().

Returns the text of the annotations Anno. If there is no text, undefined is returned.

to_term(Anno)

(since OTP 18.0)

-spec to_term(Anno) -> anno_term() when Anno :: anno().

Returns the term representing the annotations Anno.

See also from_term().

erl_eval

The Erlang meta interpreter.

This module provides an interpreter for Erlang expressions. The expressions are in the abstract syntax as returned by erl_parse, the Erlang parser, or io.

Local Function Handler

During evaluation of a function, no calls can be made to local functions. An undefined function error would be generated. However, the optional argument LocalFunctionHandler can be used to define a function that is called when there is a call to a local function. The argument can have the following formats:

{value,Func} - This defines a local function handler that is called with:

Func(Name, Arguments)

Name is the name of the local function (an atom) and Arguments is a list of the evaluated arguments. The function handler returns the value of the local function. In this case, the current bindings cannot be accessed. To signal an error, the function handler calls exit/1 with a suitable exit value.

{eval,Func} - This defines a local function handler that is called with:

Func(Name, Arguments, Bindings)

Name is the name of the local function (an atom), Arguments is a list of the unevaluated arguments, and Bindings are the current variable bindings. The function handler returns:

{value,Value,NewBindings}

Value is the value of the local function and NewBindings are the updated variable bindings. In this case, the function handler must itself evaluate all the function arguments and manage the bindings. To signal an error, the function handler calls exit/1 with a suitable exit value.

none - There is no local function handler.

Non-Local Function Handler

The optional argument NonLocalFunctionHandler can be used to define a function that is called in the following cases:

A functional object (fun) is called.

A built-in function is called.

A function is called using the M:F syntax, where M and F are atoms or expressions.

An operator Op/A is called (this is handled as a call to function erlang:Op/A).

Exceptions are calls to erlang:apply/2,3; neither of the function handlers are called for such calls. The argument can have the following formats:

{value,Func} - This defines a non-local function handler. The function may be called with two arguments:

Func(FuncSpec, Arguments)

or three arguments:

Func(Anno, FuncSpec, Arguments)

Anno is the erl_anno:anno() of the node, FuncSpec is the name of the function of the form {Module,Function} or a fun, and Arguments is a list of the evaluated arguments. The function handler returns the value of the function. To signal an error, the function handler calls exit/1 with a suitable exit value.

none - There is no non-local function handler.

Note

For calls such as erlang:apply(Fun, Args) or erlang:apply(Module, Function, Args), the call of the non-local function handler corresponding to the call to erlang:apply/2,3 itself (Func({erlang, apply}, [Fun, Args]) or Func({erlang, apply}, [Module, Function, Args])) never takes place.

The non-local function handler is however called with the evaluated arguments of the call to erlang:apply/2,3: Func(Fun, Args) or Func({Module, Function}, Args) (assuming that {Module, Function} is not {erlang, apply}).

Calls to functions defined by evaluating fun expressions "fun ... end" are also hidden from non-local function handlers.

The non-local function handler argument is probably not used as frequently as the local function handler argument. A possible use is to call exit/1 on calls to functions that for some reason are not allowed to be called.

Summary

Types

binding_struct()

A binding structure. It is either a map or an orddict. erl_eval will always return the same type as the one given.

bindings()

expression()

expression_list()

expressions()

As returned by erl_parse:parse_exprs/1 or io:parse_erl_exprs/2.

func_spec()

lfun_eval_handler()

lfun_value_handler()

local_function_handler()

Further described in section Local Function Handler in this module

name()

nlfun_handler()

non_local_function_handler()

Further described in section Non-Local Function Handler in this module.

value()

Functions

add_binding(Name, Value, BindingStruct)

Adds binding Name=Value to BindingStruct. Returns an updated binding structure.

binding(Name, BindingStruct)

Returns the binding of Name in BindingStruct.

bindings(BindingStruct)

Returns the list of bindings contained in the binding structure.

del_binding(Name, BindingStruct)

Removes the binding of Name in BindingStruct. Returns an updated binding structure.

expr(Expression, Bindings)

Equivalent to expr(Expression, Bindings, none).

expr(Expression, Bindings, LocalFunctionHandler)

Equivalent to expr(Expression, Bindings, LocalFunctionHandler, none).

expr(Expression, Bindings, LocalFunctionHandler, NonLocalFunctionHandler)

Equivalent to expr(Expression, Bindings, LocalFunctionHandler, NonLocalFunctionHandler, none).

expr(Expression, Bindings, LocalFunctionHandler, NonLocalFunctionHandler, ReturnFormat)

Evaluates Expression with the set of bindings Bindings. Expression is an expression in abstract syntax.

expr_list(ExpressionList, Bindings)

Equivalent to expr_list(ExpressionList, Bindings, none).

expr_list(ExpressionList, Bindings, LocalFunctionHandler)

Equivalent to expr_list(ExpressionList, Bindings, LocalFunctionHandler, none).

expr_list(ExpressionList, Bindings, LocalFunctionHandler, NonLocalFunctionHandler)

Evaluates a list of expressions in parallel, using the same initial bindings for each expression. Attempts are made to merge the bindings returned from each evaluation.

exprs(Expressions, Bindings)

Equivalent to exprs(Expressions, Bindings, none).

exprs(Expressions, Bindings, LocalFunctionHandler)

Equivalent to exprs(Expressions, Bindings, LocalFunctionHandler, none).

exprs(Expressions, Bindings, LocalFunctionHandler, NonLocalFunctionHandler)

Evaluates Expressions with the set of bindings Bindings, where Expressions is a sequence of expressions (in abstract syntax) of a type that can be returned by io:parse_erl_exprs/2.

new_bindings()

Returns an empty binding structure.

Types

binding_struct()

-type binding_struct() :: orddict:orddict() | map().

A binding structure. It is either a map or an orddict. erl_eval will always return the same type as the one given.

bindings()

(not exported)

-type bindings() :: [{name(), value()}].

expression()

(not exported)

-type expression() :: erl_parse:abstract_expr().

expression_list()

(not exported)

-type expression_list() :: [expression()].

expressions()

(not exported)

-type expressions() :: [erl_parse:abstract_expr()].

As returned by erl_parse:parse_exprs/1 or io:parse_erl_exprs/2.

func_spec()

(not exported)

-type func_spec() :: {Module :: module(), Function :: atom()} | function().

lfun_eval_handler()

(not exported)

-type lfun_eval_handler() ::
          fun((Name :: atom(), Arguments :: expression_list(), Bindings :: binding_struct()) ->
                  {value, Value :: value(), NewBindings :: binding_struct()}).

lfun_value_handler()

(not exported)

-type lfun_value_handler() :: fun((Name :: atom(), Arguments :: [term()]) -> Value :: value()).

local_function_handler()

(not exported)

-type local_function_handler() :: {value, lfun_value_handler()} | {eval, lfun_eval_handler()} | none.

Further described in section Local Function Handler in this module

name()

(not exported)

-type name() :: term().

nlfun_handler()

(not exported)

-type nlfun_handler() ::
          fun((FuncSpec :: func_spec(), Arguments :: [term()]) -> term()) |
          fun((Anno :: erl_anno:anno(), FuncSpec :: func_spec(), Arguments :: [term()]) -> term()).

non_local_function_handler()

(not exported)

-type non_local_function_handler() :: {value, nlfun_handler()} | none.

Further described in section Non-Local Function Handler in this module.

value()

(not exported)

-type value() :: term().

Functions

add_binding(Name, Value, BindingStruct)

-spec add_binding(Name, Value, BindingStruct) -> binding_struct()
                     when Name :: name(), Value :: value(), BindingStruct :: binding_struct().

Adds binding Name=Value to BindingStruct. Returns an updated binding structure.

binding(Name, BindingStruct)

-spec binding(Name, BindingStruct) -> {value, value()} | unbound
                 when Name :: name(), BindingStruct :: binding_struct().

Returns the binding of Name in BindingStruct.

bindings(BindingStruct)

-spec bindings(BindingStruct :: binding_struct()) -> bindings().

Returns the list of bindings contained in the binding structure.

del_binding(Name, BindingStruct)

-spec del_binding(Name, BindingStruct) -> binding_struct()
                     when Name :: name(), BindingStruct :: binding_struct().

Removes the binding of Name in BindingStruct. Returns an updated binding structure.

expr(Expression, Bindings)

-spec expr(Expression, Bindings) -> {value, Value, NewBindings}
              when
                  Expression :: expression(),
                  Bindings :: binding_struct(),
                  Value :: value(),
                  NewBindings :: binding_struct().

Equivalent to expr(Expression, Bindings, none).

expr(Expression, Bindings, LocalFunctionHandler)

-spec expr(Expression, Bindings, LocalFunctionHandler) -> {value, Value, NewBindings}
              when
                  Expression :: expression(),
                  Bindings :: binding_struct(),
                  LocalFunctionHandler :: local_function_handler(),
                  Value :: value(),
                  NewBindings :: binding_struct().

Equivalent to expr(Expression, Bindings, LocalFunctionHandler, none).

expr(Expression, Bindings, LocalFunctionHandler, NonLocalFunctionHandler)

-spec expr(Expression, Bindings, LocalFunctionHandler, NonLocalFunctionHandler) ->
              {value, Value, NewBindings}
              when
                  Expression :: expression(),
                  Bindings :: binding_struct(),
                  LocalFunctionHandler :: local_function_handler(),
                  NonLocalFunctionHandler :: non_local_function_handler(),
                  Value :: value(),
                  NewBindings :: binding_struct().

Equivalent to expr(Expression, Bindings, LocalFunctionHandler, NonLocalFunctionHandler, none).

expr(Expression, Bindings, LocalFunctionHandler, NonLocalFunctionHandler, ReturnFormat)

-spec expr(Expression, Bindings, LocalFunctionHandler, NonLocalFunctionHandler, ReturnFormat) ->
              {value, Value, NewBindings} | Value
              when
                  Expression :: expression(),
                  Bindings :: binding_struct(),
                  LocalFunctionHandler :: local_function_handler(),
                  NonLocalFunctionHandler :: non_local_function_handler(),
                  ReturnFormat :: none | value,
                  Value :: value(),
                  NewBindings :: binding_struct().

Evaluates Expression with the set of bindings Bindings. Expression is an expression in abstract syntax.

For an explanation of when and how to use arguments LocalFunctionHandler and NonLocalFunctionHandler, see sections Local Function Handler and Non-Local Function Handler in this module.

Returns {value, Value, NewBindings} by default. If ReturnFormat is value, only Value is returned.

expr_list(ExpressionList, Bindings)

-spec expr_list(ExpressionList, Bindings) -> {ValueList, NewBindings}
                   when
                       ExpressionList :: expression_list(),
                       Bindings :: binding_struct(),
                       ValueList :: [value()],
                       NewBindings :: binding_struct().

Equivalent to expr_list(ExpressionList, Bindings, none).

expr_list(ExpressionList, Bindings, LocalFunctionHandler)

-spec expr_list(ExpressionList, Bindings, LocalFunctionHandler) -> {ValueList, NewBindings}
                   when
                       ExpressionList :: expression_list(),
                       Bindings :: binding_struct(),
                       LocalFunctionHandler :: local_function_handler(),
                       ValueList :: [value()],
                       NewBindings :: binding_struct().

Equivalent to expr_list(ExpressionList, Bindings, LocalFunctionHandler, none).

expr_list(ExpressionList, Bindings, LocalFunctionHandler, NonLocalFunctionHandler)

-spec expr_list(ExpressionList, Bindings, LocalFunctionHandler, NonLocalFunctionHandler) ->
                   {ValueList, NewBindings}
                   when
                       ExpressionList :: expression_list(),
                       Bindings :: binding_struct(),
                       LocalFunctionHandler :: local_function_handler(),
                       NonLocalFunctionHandler :: non_local_function_handler(),
                       ValueList :: [value()],
                       NewBindings :: binding_struct().

Evaluates a list of expressions in parallel, using the same initial bindings for each expression. Attempts are made to merge the bindings returned from each evaluation.

This function is useful in LocalFunctionHandler, see section Local Function Handler in this module.

Returns {ValueList, NewBindings}.

exprs(Expressions, Bindings)

-spec exprs(Expressions, Bindings) -> {value, Value, NewBindings}
               when
                   Expressions :: expressions(),
                   Bindings :: binding_struct(),
                   Value :: value(),
                   NewBindings :: binding_struct().

Equivalent to exprs(Expressions, Bindings, none).

exprs(Expressions, Bindings, LocalFunctionHandler)

-spec exprs(Expressions, Bindings, LocalFunctionHandler) -> {value, Value, NewBindings}
               when
                   Expressions :: expressions(),
                   Bindings :: binding_struct(),
                   LocalFunctionHandler :: local_function_handler(),
                   Value :: value(),
                   NewBindings :: binding_struct().

Equivalent to exprs(Expressions, Bindings, LocalFunctionHandler, none).

exprs(Expressions, Bindings, LocalFunctionHandler, NonLocalFunctionHandler)

-spec exprs(Expressions, Bindings, LocalFunctionHandler, NonLocalFunctionHandler) ->
               {value, Value, NewBindings}
               when
                   Expressions :: expressions(),
                   Bindings :: binding_struct(),
                   LocalFunctionHandler :: local_function_handler(),
                   NonLocalFunctionHandler :: non_local_function_handler(),
                   Value :: value(),
                   NewBindings :: binding_struct().

Evaluates Expressions with the set of bindings Bindings, where Expressions is a sequence of expressions (in abstract syntax) of a type that can be returned by io:parse_erl_exprs/2.

For an explanation of when and how to use arguments LocalFunctionHandler and NonLocalFunctionHandler, see sections Local Function Handler and Non-Local Function Handler in this module.

Returns {value, Value, NewBindings}

new_bindings()

-spec new_bindings() -> binding_struct().

Returns an empty binding structure.

erl_expand_records

This module expands records in a module.

See Also

Section The Abstract Format in ERTS User's Guide.

Summary

Functions

module(AbsForms, CompileOptions)

Expands all records in a module to use explicit tuple operations and adds explicit module names to calls to BIFs and imported functions. The returned module has no references to records, attributes, or code.

Functions

module(AbsForms, CompileOptions)

-spec module(AbsForms, CompileOptions) -> AbsForms2
                when
                    AbsForms :: [erl_parse:abstract_form()],
                    AbsForms2 :: [erl_parse:abstract_form()],
                    CompileOptions :: [compile:option()].

Expands all records in a module to use explicit tuple operations and adds explicit module names to calls to BIFs and imported functions. The returned module has no references to records, attributes, or code.

erl_features

This module contains functions for supporting features that can be enabled/disabled in Erlang.

It should be considered as mostly for internal use, although there are some functions that might be useful when writing tools.

Summary

Types

feature()

release()

status()

type()

Functions

all()

Return a list of all known features. This list will include features that have been removed (status rejected) and features that are no longer configurable (status permanent).

configurable()

Return a list of all configurable features, that is, features with status experimental or approved. These are the features that can be enabled or disabled.

enabled()

Return a list of the features that are currently enabled. Note that the set of enabled is set during startup and can then not be changed.

info(Feature)

Return a map containing information about the given feature.

used/1

Return the list of features enabled when compiling the module. The module need not be loaded, but is found if it exists in the loadpath. If not all features used by the module are enabled in the runtime, loading the module is not allowed.

Types

feature()

(not exported) (since OTP 25.0)

-type feature() :: atom().

release()

(not exported) (since OTP 25.0)

-type release() :: non_neg_integer().

status()

(not exported) (since OTP 25.0)

-type status() :: experimental | approved | permanent | rejected.

type()

(not exported) (since OTP 25.0)

-type type() :: extension | backwards_incompatible_change.

Functions

all()

(since OTP 25.0)

-spec all() -> [feature()].

Return a list of all known features. This list will include features that have been removed (status rejected) and features that are no longer configurable (status permanent).

configurable()

(since OTP 25.1)

-spec configurable() -> [feature()].

Return a list of all configurable features, that is, features with status experimental or approved. These are the features that can be enabled or disabled.

enabled()

(since OTP 25.0)

-spec enabled() -> [feature()].

Return a list of the features that are currently enabled. Note that the set of enabled is set during startup and can then not be changed.

info(Feature)

(since OTP 25.0)

-spec info(feature()) -> FeatureInfoMap | no_return()
              when
                  Description :: string(),
                  FeatureInfoMap ::
                      #{description := Description,
                        short := Description,
                        type := type(),
                        keywords := [atom()],
                        status := status(),
                        experimental => release(),
                        approved => release(),
                        permanent => release(),
                        rejected => release()}.

Return a map containing information about the given feature.

used/1

(since OTP 25.0)

-spec used(module() | file:filename()) -> [feature()].

Return the list of features enabled when compiling the module. The module need not be loaded, but is found if it exists in the loadpath. If not all features used by the module are enabled in the runtime, loading the module is not allowed.

erl_id_trans

This module performs an identity parse transformation of Erlang code.

It is included as an example for users who wants to write their own parse transformers. If option {parse_transform,Module} is passed to the compiler, a user-written function parse_transform/2 is called by the compiler before the code is checked for errors.

Before the function parse_transform/2 is called, the Erlang Compiler checks if the parse transformation can handle abstract code with column numbers: If the function parse_transform_info/0 is implemented and returns a map where the key error_location is associated with the value line, the compiler removes column numbers from the abstract code before calling the parse transform. Otherwise, the compiler passes the abstract code on without modification.

Parse Transformations

Parse transformations are used if a programmer wants to use Erlang syntax, but with different semantics. The original Erlang code is then transformed into other Erlang code.

Note

Programmers are strongly advised not to engage in parse transformations. No support is offered for problems encountered.

See Also

erl_parse and compile.

Summary

Functions

parse_transform(Forms, Options)

Performs an identity transformation on Erlang forms, as an example.

parse_transform_info()

Returns information about the parse transform itself.

Functions

parse_transform(Forms, Options)

-spec parse_transform(Forms, Options) -> NewForms
                         when
                             Forms :: [erl_parse:abstract_form() | erl_parse:form_info()],
                             NewForms :: Forms,
                             Options :: [compile:option()].

Performs an identity transformation on Erlang forms, as an example.

parse_transform_info()

(since OTP 24.0)

-spec parse_transform_info() -> #{error_location => column | line}.

Returns information about the parse transform itself.

erl_internal

Internal Erlang definitions.

This module defines Erlang BIFs, guard tests, and operators. This module is only of interest to programmers who manipulate Erlang code.

Summary

Functions

add_predefined_functions(Forms)

Adds to Forms the code for the standard pre-defined functions (such as module_info/0) that are to be included in every module.

arith_op(OpName, Arity)

Returns true if OpName/Arity is an arithmetic operator, otherwise false.

bif(Name, Arity)

Returns true if Name/Arity is an Erlang BIF that is automatically recognized by the compiler, otherwise false.

bool_op(OpName, Arity)

Returns true if OpName/Arity is a Boolean operator, otherwise false.

comp_op(OpName, Arity)

Returns true if OpName/Arity is a comparison operator, otherwise false.

guard_bif(Name, Arity)

Returns true if Name/Arity is an Erlang BIF that is allowed in guards, otherwise false.

list_op(OpName, Arity)

Returns true if OpName/Arity is a list operator, otherwise false.

op_type(OpName, Arity)

Returns the Type of operator that OpName/Arity belongs to, or generates a function_clause error if it is not an operator.

send_op(OpName, Arity)

Returns true if OpName/Arity is a send operator, otherwise false.

type_test(Name, Arity)

Returns true if Name/Arity is a valid Erlang type test, otherwise false.

Functions

add_predefined_functions(Forms)

(since OTP 20.0)

-spec add_predefined_functions(Forms) -> UpdatedForms
                                  when
                                      Forms :: [erl_parse:abstract_form() | erl_parse:form_info()],
                                      UpdatedForms ::
                                          [erl_parse:abstract_form() | erl_parse:form_info()].

Adds to Forms the code for the standard pre-defined functions (such as module_info/0) that are to be included in every module.

arith_op(OpName, Arity)

-spec arith_op(OpName, Arity) -> boolean() when OpName :: atom(), Arity :: arity().

Returns true if OpName/Arity is an arithmetic operator, otherwise false.

bif(Name, Arity)

-spec bif(Name, Arity) -> boolean() when Name :: atom(), Arity :: arity().

Returns true if Name/Arity is an Erlang BIF that is automatically recognized by the compiler, otherwise false.

bool_op(OpName, Arity)

-spec bool_op(OpName, Arity) -> boolean() when OpName :: atom(), Arity :: arity().

Returns true if OpName/Arity is a Boolean operator, otherwise false.

comp_op(OpName, Arity)

-spec comp_op(OpName, Arity) -> boolean() when OpName :: atom(), Arity :: arity().

Returns true if OpName/Arity is a comparison operator, otherwise false.

guard_bif(Name, Arity)

-spec guard_bif(Name, Arity) -> boolean() when Name :: atom(), Arity :: arity().

Returns true if Name/Arity is an Erlang BIF that is allowed in guards, otherwise false.

list_op(OpName, Arity)

-spec list_op(OpName, Arity) -> boolean() when OpName :: atom(), Arity :: arity().

Returns true if OpName/Arity is a list operator, otherwise false.

op_type(OpName, Arity)

-spec op_type(OpName, Arity) -> Type
                 when OpName :: atom(), Arity :: arity(), Type :: arith | bool | comp | list | send.

Returns the Type of operator that OpName/Arity belongs to, or generates a function_clause error if it is not an operator.

send_op(OpName, Arity)

-spec send_op(OpName, Arity) -> boolean() when OpName :: atom(), Arity :: arity().

Returns true if OpName/Arity is a send operator, otherwise false.

type_test(Name, Arity)

-spec type_test(Name, Arity) -> boolean() when Name :: atom(), Arity :: arity().

Returns true if Name/Arity is a valid Erlang type test, otherwise false.

erl_lint

The Erlang code linter.

This module is used to check Erlang code for illegal syntax and other bugs. It also warns against coding practices that are not recommended.

The errors detected include:

Redefined and undefined functions

Unbound and unsafe variables

Illegal record use

The warnings detected include:

Unused functions and imports

Unused variables

Variables imported into matches

Variables exported from if/case/receive

Variables shadowed in funs and list comprehensions

Some of the warnings are optional, and can be turned on by specifying the appropriate option, described below.

The functions in this module are invoked automatically by the Erlang compiler. There is no reason to invoke these functions separately unless you have written your own Erlang compiler.

Error Information

ErrorInfo is the standard ErrorInfo structure that is returned from all I/O modules. The format is as follows:

{ErrorLine, Module, ErrorDescriptor}

A string describing the error is obtained with the following call:

Module:format_error(ErrorDescriptor)

See Also

epp, erl_parse

Summary

Types

error_description()

error_info()

fa()

fun_used_vars()

Functions

format_error(ErrorDescriptor)

Takes an ErrorDescriptor and returns a string that describes the error or warning. This function is usually called implicitly when processing an ErrorInfo structure (see section Error Information).

is_guard_test(Expr)

Tests if Expr is a legal guard test. Expr is an Erlang term representing the abstract form for the expression. erl_parse:parse_exprs(Tokens) can be used to generate a list of Expr.

module(AbsForms)

Equivalent to module/3.

module(AbsForms, FileName)

Equivalent to module/3.

module(AbsForms, FileName, CompileOptions)

Checks all the forms in a module for errors. It returns

Types

error_description()

-type error_description() :: term().

error_info()

-type error_info() :: {erl_anno:location() | none, module(), error_description()}.

fa()

(not exported)

-type fa() :: {atom(), arity()}.

fun_used_vars()

-type fun_used_vars() :: #{erl_parse:abstract_expr() => {[atom()], fun_used_vars()}}.

Functions

format_error(ErrorDescriptor)

-spec format_error(ErrorDescriptor) -> io_lib:chars() when ErrorDescriptor :: error_description().

Takes an ErrorDescriptor and returns a string that describes the error or warning. This function is usually called implicitly when processing an ErrorInfo structure (see section Error Information).

is_guard_test(Expr)

-spec is_guard_test(Expr) -> boolean() when Expr :: erl_parse:abstract_expr().

Tests if Expr is a legal guard test. Expr is an Erlang term representing the abstract form for the expression. erl_parse:parse_exprs(Tokens) can be used to generate a list of Expr.

module(AbsForms)

-spec module(AbsForms) -> {ok, Warnings} | {error, Errors, Warnings}
                when
                    AbsForms :: [erl_parse:abstract_form() | erl_parse:form_info()],
                    Warnings :: [{SourceFile, [ErrorInfo]}],
                    Errors :: [{SourceFile, [ErrorInfo]}],
                    SourceFile :: file:filename(),
                    ErrorInfo :: error_info().

Equivalent to module/3.

module(AbsForms, FileName)

-spec module(AbsForms, FileName) -> {ok, Warnings} | {error, Errors, Warnings}
                when
                    AbsForms :: [erl_parse:abstract_form() | erl_parse:form_info()],
                    FileName :: atom() | string(),
                    Warnings :: [{SourceFile, [ErrorInfo]}],
                    Errors :: [{SourceFile, [ErrorInfo]}],
                    SourceFile :: file:filename(),
                    ErrorInfo :: error_info().

Equivalent to module/3.

module(AbsForms, FileName, CompileOptions)

-spec module(AbsForms, FileName, CompileOptions) -> {ok, Warnings} | {error, Errors, Warnings}
                when
                    AbsForms :: [erl_parse:abstract_form() | erl_parse:form_info()],
                    FileName :: atom() | string(),
                    CompileOptions :: [compile:option()],
                    Warnings :: [{SourceFile, [ErrorInfo]}],
                    Errors :: [{SourceFile, [ErrorInfo]}],
                    SourceFile :: file:filename(),
                    ErrorInfo :: error_info().

Checks all the forms in a module for errors. It returns:

{ok,Warnings} - There are no errors in the module.

{error,Errors,Warnings} - There are errors in the module.

As this module is of interest only to the maintainers of the compiler, and to avoid the same description in two places, the elements of Options that control the warnings are only described in the compile module.

AbsForms of a module, which comes from a file that is read through epp, the Erlang preprocessor, can come from many files. This means that any references to errors must include the filename, see the epp module or parser (see the erl_parse module). The returned errors and warnings have the following format:

[{SourceFile,[ErrorInfo]}]

The errors and warnings are listed in the order in which they are encountered in the forms. The errors from one file can therefore be split into different entries in the list of errors.

erl_parse

This module is the basic Erlang parser that converts tokens into the abstract form of either forms (that is, top-level constructs), expressions, or terms.

The Abstract Format is described in the ERTS User's Guide. Notice that a token list must end with the dot token to be acceptable to the parse functions (see the erl_scan) module.

Error Information

ErrorInfo is the standard ErrorInfo structure that is returned from all I/O modules. The format is as follows:

{ErrorLine, Module, ErrorDescriptor}

A string describing the error is obtained with the following call:

Module:format_error(ErrorDescriptor)

See Also

erl_anno, erl_scan, io, section The Abstract Format in the ERTS User's Guide.

Summary

Types

abstract_clause()

Abstract form of an Erlang clause.

abstract_expr()

Abstract form of an Erlang expression.

abstract_form()

Abstract form of an Erlang form.

abstract_type()

Abstract form of an Erlang type.

af_anno()

af_annotated_type()

af_args()

af_assoc(T)

af_assoc_exact(T)

af_assoc_type()

af_atom()

af_behavior()

af_behaviour()

af_bin(T)

af_binary_comprehension()

af_binary_op(T)

af_binelement(T)

Abstract representation of an element of a bitstring.

af_binelement_size()

af_bitstring_type()

af_block()

af_body()

af_case()

af_catch()

af_character()

af_clause()

af_clause_seq()

af_compile()

af_cons(T)

af_constrained_function_type()

af_constraint()

af_empty_list_type()

af_export()

af_export_type()

af_fa_list()

af_field()

af_field_decl()

Abstract representation of a record field.

af_field_name()

af_file()

af_filter()

af_float()

af_fun()

af_fun_type()

af_function_constraint()

af_function_decl()

af_function_spec()

af_function_type()

af_function_type_list()

af_generator()

Abstract representation of a list, bitstring or map generator.

af_guard()

af_guard_call()

af_guard_seq()

af_guard_test()

af_if()

af_import()

af_integer()

af_integer_range_type()

af_list_comprehension()

af_lit_atom(A)

af_literal()

af_local_call()

af_local_fun()

af_local_function()

af_map_comprehension()

af_map_creation(T)

af_map_pattern()

af_map_type()

af_map_update(T)

af_match(T)

af_maybe()

af_maybe_else()

af_maybe_match()

af_module()

af_named_fun()

af_nil()

af_pattern()

af_predefined_type()

af_qualifier()

af_qualifier_seq()

af_receive()

af_record_creation(T)

af_record_decl()

af_record_field(T)

af_record_field_access(T)

af_record_field_type()

af_record_index()

af_record_type()

af_record_update(T)

af_remote_call()

af_remote_fun()

af_remote_function()

Abstract representation of a remote function call.

af_remote_guard_call()

af_remote_type()

af_singleton_integer_type()

af_string()

af_ta_list()

af_template()

af_try()

af_tuple(T)

af_tuple_type()

af_type_decl()

af_type_union()

af_type_variable()

af_typed_field()

af_unary_op(T)

af_user_defined_type()

af_variable()

af_wild_attribute()

af_zip_generator()

anno()

behaviour()

binary_op()

encoding_func()

endianness()

erl_parse_tree()

error_description()

error_info()

form_info()

Tuples {error, error_info()} and {warning, error_info()}, denoting syntactically incorrect forms and warnings, and {eof, line()}, denoting an end-of-stream encountered before a complete form had been parsed.

fun_name()

function_name()

record_name()

signedness()

spec_attr()

token()

type()

type_attr()

type_name()

type_specifier()

type_specifier_list()

unary_op()

unit()

Functions

abstract(Data)

Converts the Erlang data structure Data into an abstract form of type AbsTerm. This function is the inverse of normalise/1.

abstract(Data, Options)

Converts the Erlang data structure Data into an abstract form of type AbsTerm.

anno_from_term(Term)

Assumes that Term is a term with the same structure as a erl_parse tree, but with terms, say T, where a erl_parse tree has collections of annotations.

anno_to_term(Abstr)

Returns a term where each collection of annotations Anno of the nodes of the erl_parse tree Abstr is replaced by the term returned by erl_anno:to_term(Anno). The erl_parse tree is traversed in a depth-first, left-to-right fashion.

fold_anno(Fun, Acc0, Abstr)

Updates an accumulator by applying Fun on each collection of annotations of the erl_parse tree Abstr.

format_error(Message)

Uses an ErrorDescriptor and returns a string that describes the error.

map_anno(Fun, Abstr)

Modifies the erl_parse tree Abstr by applying Fun on each collection of annotations of the nodes of the erl_parse tree. The erl_parse tree is traversed in a depth-first, left-to-right fashion.

mapfold_anno(Fun, Acc0, Abstr)

Modifies the erl_parse tree Abstr by applying Fun on each collection of annotations of the nodes of the erl_parse tree, while at the same time updating an accumulator.

new_anno(Term)

Assumes that Term is a term with the same structure as a erl_parse tree, but with locations where a erl_parse tree has collections of annotations.

normalise(AbsTerm)

Converts the abstract form AbsTerm of a term into a conventional Erlang data structure (that is, the term itself). This function is the inverse of abstract/1.

parse_exprs(Tokens)

Parses Tokens as if it was a list of expressions.

parse_form(Tokens)

Parses Tokens as if it was a form.

parse_term(Tokens)

Parses Tokens as if it was a term.

tokens(AbsTerm)

Equivalent to tokens(AbsTerm, []).

tokens(AbsTerm, MoreTokens)

Generates a list of tokens representing the abstract form AbsTerm of an expression. Optionally, MoreTokens is appended.

Types

abstract_clause()

-type abstract_clause() :: af_clause().

Abstract form of an Erlang clause.

abstract_expr()

-type abstract_expr() ::
          af_literal() |
          af_match(abstract_expr()) |
          af_maybe_match() |
          af_variable() |
          af_tuple(abstract_expr()) |
          af_nil() |
          af_cons(abstract_expr()) |
          af_bin(abstract_expr()) |
          af_binary_op(abstract_expr()) |
          af_unary_op(abstract_expr()) |
          af_record_creation(abstract_expr()) |
          af_record_update(abstract_expr()) |
          af_record_index() |
          af_record_field_access(abstract_expr()) |
          af_map_creation(abstract_expr()) |
          af_map_update(abstract_expr()) |
          af_catch() |
          af_local_call() |
          af_remote_call() |
          af_list_comprehension() |
          af_map_comprehension() |
          af_binary_comprehension() |
          af_block() |
          af_if() |
          af_case() |
          af_try() |
          af_receive() |
          af_local_fun() |
          af_remote_fun() |
          af_fun() |
          af_named_fun() |
          af_maybe() |
          af_maybe_else().

Abstract form of an Erlang expression.

abstract_form()

-type abstract_form() ::
          af_module() |
          af_behavior() |
          af_behaviour() |
          af_export() |
          af_import() |
          af_export_type() |
          af_compile() |
          af_file() |
          af_record_decl() |
          af_type_decl() |
          af_function_spec() |
          af_wild_attribute() |
          af_function_decl().

Abstract form of an Erlang form.

abstract_type()

-type abstract_type() ::
          af_annotated_type() |
          af_atom() |
          af_bitstring_type() |
          af_empty_list_type() |
          af_fun_type() |
          af_integer_range_type() |
          af_map_type() |
          af_predefined_type() |
          af_record_type() |
          af_remote_type() |
          af_singleton_integer_type() |
          af_tuple_type() |
          af_type_union() |
          af_type_variable() |
          af_user_defined_type().

Abstract form of an Erlang type.

af_anno()

(not exported)

-type af_anno() :: af_variable().

af_annotated_type()

(not exported)

-type af_annotated_type() :: {ann_type, anno(), [af_anno() | abstract_type()]}.

af_args()

(not exported)

-type af_args() :: [abstract_expr()].

af_assoc(T)

(not exported)

-type af_assoc(T) :: {map_field_assoc, anno(), T, T} | af_assoc_exact(T).

af_assoc_exact(T)

(not exported)

-type af_assoc_exact(T) :: {map_field_exact, anno(), T, T}.

af_assoc_type()

(not exported)

-type af_assoc_type() ::
          {type, anno(), map_field_assoc, [abstract_type()]} |
          {type, anno(), map_field_exact, [abstract_type()]}.

af_atom()

(not exported)

-type af_atom() :: af_lit_atom(atom()).

af_behavior()

(not exported)

-type af_behavior() :: {attribute, anno(), behavior, behaviour()}.

af_behaviour()

(not exported)

-type af_behaviour() :: {attribute, anno(), behaviour, behaviour()}.

af_bin(T)

(not exported)

-type af_bin(T) :: {bin, anno(), [af_binelement(T)]}.

af_binary_comprehension()

(not exported)

-type af_binary_comprehension() :: {bc, anno(), af_template(), af_qualifier_seq()}.

af_binary_op(T)

(not exported)

-type af_binary_op(T) :: {op, anno(), binary_op(), T, T}.

af_binelement(T)

-type af_binelement(T) :: {bin_element, anno(), T, af_binelement_size(), type_specifier_list()}.

Abstract representation of an element of a bitstring.

af_binelement_size()

(not exported)

-type af_binelement_size() :: default | abstract_expr().

af_bitstring_type()

(not exported)

-type af_bitstring_type() :: {type, anno(), binary, [af_singleton_integer_type()]}.

af_block()

(not exported)

-type af_block() :: {block, anno(), af_body()}.

af_body()

(not exported)

-type af_body() :: [abstract_expr(), ...].

af_case()

(not exported)

-type af_case() :: {'case', anno(), abstract_expr(), af_clause_seq()}.

af_catch()

(not exported)

-type af_catch() :: {'catch', anno(), abstract_expr()}.

af_character()

(not exported)

-type af_character() :: {char, anno(), char()}.

af_clause()

(not exported)

-type af_clause() :: {clause, anno(), [af_pattern()], af_guard_seq(), af_body()}.

af_clause_seq()

(not exported)

-type af_clause_seq() :: [af_clause(), ...].

af_compile()

(not exported)

-type af_compile() :: {attribute, anno(), compile, any()}.

af_cons(T)

(not exported)

-type af_cons(T) :: {cons, anno(), T, T}.

af_constrained_function_type()

(not exported)

-type af_constrained_function_type() ::
          {type, anno(), bounded_fun, [af_function_type() | af_function_constraint()]}.

af_constraint()

(not exported)

-type af_constraint() ::
          {type, anno(), constraint, [af_lit_atom(is_subtype) | [af_type_variable() | abstract_type()]]}.

af_empty_list_type()

(not exported)

-type af_empty_list_type() :: {type, anno(), nil, []}.

af_export()

(not exported)

-type af_export() :: {attribute, anno(), export, af_fa_list()}.

af_export_type()

(not exported)

-type af_export_type() :: {attribute, anno(), export_type, af_ta_list()}.

af_fa_list()

(not exported)

-type af_fa_list() :: [{function_name(), arity()}].

af_field()

(not exported)

-type af_field() ::
          {record_field, anno(), af_field_name()} |
          {record_field, anno(), af_field_name(), abstract_expr()}.

af_field_decl()

-type af_field_decl() :: af_typed_field() | af_field().

Abstract representation of a record field.

af_field_name()

(not exported)

-type af_field_name() :: af_atom().

af_file()

(not exported)

-type af_file() :: {attribute, anno(), file, {string(), anno()}}.

af_filter()

(not exported)

-type af_filter() :: abstract_expr().

af_float()

(not exported)

-type af_float() :: {float, anno(), float()}.

af_fun()

(not exported)

-type af_fun() :: {'fun', anno(), {clauses, af_clause_seq()}}.

af_fun_type()

(not exported)

-type af_fun_type() ::
          {type, anno(), 'fun', []} |
          {type, anno(), 'fun', [{type, anno(), any} | abstract_type()]} |
          af_function_type().

af_function_constraint()

(not exported)

-type af_function_constraint() :: [af_constraint(), ...].

af_function_decl()

(not exported)

-type af_function_decl() :: {function, anno(), function_name(), arity(), af_clause_seq()}.

af_function_spec()

(not exported)

-type af_function_spec() ::
          {attribute, anno(), spec_attr(), {{function_name(), arity()}, af_function_type_list()}} |
          {attribute, anno(), spec, {{module(), function_name(), arity()}, af_function_type_list()}}.

af_function_type()

(not exported)

-type af_function_type() ::
          {type, anno(), 'fun', [{type, anno(), product, [abstract_type()]} | abstract_type()]}.

af_function_type_list()

(not exported)

-type af_function_type_list() :: [af_constrained_function_type() | af_function_type(), ...].

af_generator()

-type af_generator() ::
          {generate, anno(), af_pattern(), abstract_expr()} |
          {generate_strict, anno(), af_pattern(), abstract_expr()} |
          {m_generate, anno(), af_assoc_exact(af_pattern()), abstract_expr()} |
          {m_generate_strict, anno(), af_assoc_exact(af_pattern()), abstract_expr()} |
          {b_generate, anno(), af_pattern(), abstract_expr()} |
          {b_generate_strict, anno(), af_pattern(), abstract_expr()} |
          af_zip_generator().

Abstract representation of a list, bitstring or map generator.

af_guard()

(not exported)

-type af_guard() :: [af_guard_test(), ...].

af_guard_call()

(not exported)

-type af_guard_call() :: {call, anno(), af_atom(), [af_guard_test()]}.

af_guard_seq()

(not exported)

-type af_guard_seq() :: [af_guard()].

af_guard_test()

(not exported)

-type af_guard_test() ::
          af_literal() |
          af_variable() |
          af_tuple(af_guard_test()) |
          af_nil() |
          af_cons(af_guard_test()) |
          af_bin(af_guard_test()) |
          af_binary_op(af_guard_test()) |
          af_unary_op(af_guard_test()) |
          af_record_creation(af_guard_test()) |
          af_record_index() |
          af_record_field_access(af_guard_test()) |
          af_map_creation(af_guard_test()) |
          af_map_update(af_guard_test()) |
          af_guard_call() |
          af_remote_guard_call().

af_if()

(not exported)

-type af_if() :: {'if', anno(), af_clause_seq()}.

af_import()

(not exported)

-type af_import() :: {attribute, anno(), import, {module(), af_fa_list()}}.

af_integer()

(not exported)

-type af_integer() :: {integer, anno(), non_neg_integer()}.

af_integer_range_type()

(not exported)

-type af_integer_range_type() :: {type, anno(), range, [af_singleton_integer_type()]}.

af_list_comprehension()

(not exported)

-type af_list_comprehension() :: {lc, anno(), af_template(), af_qualifier_seq()}.

af_lit_atom(A)

(not exported)

-type af_lit_atom(A) :: {atom, anno(), A}.

af_literal()

(not exported)

-type af_literal() :: af_atom() | af_character() | af_float() | af_integer() | af_string().

af_local_call()

(not exported)

-type af_local_call() :: {call, anno(), af_local_function(), af_args()}.

af_local_fun()

(not exported)

-type af_local_fun() :: {'fun', anno(), {function, function_name(), arity()}}.

af_local_function()

(not exported)

-type af_local_function() :: abstract_expr().

af_map_comprehension()

(not exported)

-type af_map_comprehension() :: {mc, anno(), af_assoc(abstract_expr()), af_qualifier_seq()}.

af_map_creation(T)

(not exported)

-type af_map_creation(T) :: {map, anno(), [af_assoc(T)]}.

af_map_pattern()

(not exported)

-type af_map_pattern() :: {map, anno(), [af_assoc_exact(af_pattern())]}.

af_map_type()

(not exported)

-type af_map_type() :: {type, anno(), map, any} | {type, anno(), map, [af_assoc_type()]}.

af_map_update(T)

(not exported)

-type af_map_update(T) :: {map, anno(), T, [af_assoc(T)]}.

af_match(T)

(not exported)

-type af_match(T) :: {match, anno(), af_pattern(), T}.

af_maybe()

(not exported)

-type af_maybe() :: {'maybe', anno(), af_body()}.

af_maybe_else()

(not exported)

-type af_maybe_else() :: {'maybe', anno(), af_body(), {'else', anno(), af_clause_seq()}}.

af_maybe_match()

(not exported)

-type af_maybe_match() :: {maybe_match, anno(), af_pattern(), abstract_expr()}.

af_module()

(not exported)

-type af_module() :: {attribute, anno(), module, module()}.

af_named_fun()

(not exported)

-type af_named_fun() :: {named_fun, anno(), fun_name(), af_clause_seq()}.

af_nil()

(not exported)

-type af_nil() :: {nil, anno()}.

af_pattern()

(not exported)

-type af_pattern() ::
          af_literal() |
          af_match(af_pattern()) |
          af_variable() |
          af_tuple(af_pattern()) |
          af_nil() |
          af_cons(af_pattern()) |
          af_bin(af_pattern()) |
          af_binary_op(af_pattern()) |
          af_unary_op(af_pattern()) |
          af_record_creation(af_pattern()) |
          af_record_index() |
          af_map_pattern().

af_predefined_type()

(not exported)

-type af_predefined_type() :: {type, anno(), type_name(), [abstract_type()]}.

af_qualifier()

(not exported)

-type af_qualifier() :: af_generator() | af_filter().

af_qualifier_seq()

(not exported)

-type af_qualifier_seq() :: [af_qualifier(), ...].

af_receive()

(not exported)

-type af_receive() ::
          {'receive', anno(), af_clause_seq()} |
          {'receive', anno(), af_clause_seq(), abstract_expr(), af_body()}.

af_record_creation(T)

(not exported)

-type af_record_creation(T) :: {record, anno(), record_name(), [af_record_field(T)]}.

af_record_decl()

(not exported)

-type af_record_decl() :: {attribute, anno(), record, {record_name(), [af_field_decl()]}}.

af_record_field(T)

(not exported)

-type af_record_field(T) :: {record_field, anno(), af_field_name(), T}.

af_record_field_access(T)

(not exported)

-type af_record_field_access(T) :: {record_field, anno(), T, record_name(), af_field_name()}.

af_record_field_type()

(not exported)

-type af_record_field_type() :: {type, anno(), field_type, [(Name :: af_atom()) | abstract_type()]}.

af_record_index()

(not exported)

-type af_record_index() :: {record_index, anno(), record_name(), af_field_name()}.

af_record_type()

(not exported)

-type af_record_type() :: {type, anno(), record, [(Name :: af_atom()) | af_record_field_type()]}.

af_record_update(T)

(not exported)

-type af_record_update(T) :: {record, anno(), abstract_expr(), record_name(), [af_record_field(T)]}.

af_remote_call()

(not exported)

-type af_remote_call() :: {call, anno(), af_remote_function(), af_args()}.

af_remote_fun()

(not exported)

-type af_remote_fun() ::
          {'fun', anno(), {function, module(), function_name(), arity()}} |
          {'fun',
           anno(),
           {function,
            af_atom() | af_variable(),
            af_atom() | af_variable(),
            af_integer() | af_variable()}}.

af_remote_function()

-type af_remote_function() :: {remote, anno(), abstract_expr(), abstract_expr()}.

Abstract representation of a remote function call.

af_remote_guard_call()

(not exported)

-type af_remote_guard_call() ::
          {call, anno(), {remote, anno(), af_lit_atom(erlang), af_atom()}, [af_guard_test()]}.

af_remote_type()

(not exported)

-type af_remote_type() ::
          {remote_type, anno(), [(Module :: af_atom()) | (TypeName :: af_atom()) | [abstract_type()]]}.

af_singleton_integer_type()

(not exported)

-type af_singleton_integer_type() ::
          af_integer() |
          af_character() |
          af_unary_op(af_singleton_integer_type()) |
          af_binary_op(af_singleton_integer_type()).

af_string()

(not exported)

-type af_string() :: {string, anno(), string()}.

af_ta_list()

(not exported)

-type af_ta_list() :: [{type_name(), arity()}].

af_template()

(not exported)

-type af_template() :: abstract_expr().

af_try()

(not exported)

-type af_try() :: {'try', anno(), af_body(), af_clause_seq() | [], af_clause_seq() | [], af_body() | []}.

af_tuple(T)

(not exported)

-type af_tuple(T) :: {tuple, anno(), [T]}.

af_tuple_type()

(not exported)

-type af_tuple_type() :: {type, anno(), tuple, any} | {type, anno(), tuple, [abstract_type()]}.

af_type_decl()

(not exported)

-type af_type_decl() ::
          {attribute, anno(), type_attr(), {type_name(), abstract_type(), [af_variable()]}}.

af_type_union()

(not exported)

-type af_type_union() :: {type, anno(), union, [abstract_type(), ...]}.

af_type_variable()

(not exported)

-type af_type_variable() :: {var, anno(), atom()}.

af_typed_field()

(not exported)

-type af_typed_field() :: {typed_record_field, af_field(), abstract_type()}.

af_unary_op(T)

(not exported)

-type af_unary_op(T) :: {op, anno(), unary_op(), T}.

af_user_defined_type()

(not exported)

-type af_user_defined_type() :: {user_type, anno(), type_name(), [abstract_type()]}.

af_variable()

(not exported)

-type af_variable() :: {var, anno(), atom()}.

af_wild_attribute()

(not exported)

-type af_wild_attribute() :: {attribute, anno(), atom(), any()}.

af_zip_generator()

-type af_zip_generator() :: {zip, anno(), [af_generator(), ...]}.

anno()

(not exported)

-type anno() :: erl_anno:anno().

behaviour()

(not exported)

-type behaviour() :: atom().

binary_op()

(not exported)

-type binary_op() ::
          '/' | '*' | 'div' | 'rem' | 'band' | 'and' | '+' | '-' | 'bor' | 'bxor' | 'bsl' | 'bsr' |
          'or' | 'xor' | '++' | '--' | '==' | '/=' | '=<' | '<' | '>=' | '>' | '=:=' | '=/=' | '!' |
          'andalso' | 'orelse'.

encoding_func()

(not exported)

-type encoding_func() :: fun((non_neg_integer()) -> boolean()).

endianness()

(not exported)

-type endianness() :: big | little | native.

erl_parse_tree()

(not exported)

-type erl_parse_tree() :: abstract_clause() | abstract_expr() | abstract_form() | abstract_type().

error_description()

(not exported)

-type error_description() :: term().

error_info()

-type error_info() :: {erl_anno:location(), module(), error_description()}.

form_info()

-type form_info() ::
          {eof, erl_anno:location()} |
          {error, erl_scan:error_info() | error_info()} |
          {warning, erl_scan:error_info() | error_info()}.

Tuples {error, error_info()} and {warning, error_info()}, denoting syntactically incorrect forms and warnings, and {eof, line()}, denoting an end-of-stream encountered before a complete form had been parsed.

fun_name()

(not exported)

-type fun_name() :: atom().

function_name()

(not exported)

-type function_name() :: atom().

record_name()

(not exported)

-type record_name() :: atom().

signedness()

(not exported)

-type signedness() :: signed | unsigned.

spec_attr()

(not exported)

-type spec_attr() :: callback | spec.

token()

(not exported)

-type token() :: erl_scan:token().

type()

(not exported)

-type type() :: integer | float | binary | bytes | bitstring | bits | utf8 | utf16 | utf32.

type_attr()

(not exported)

-type type_attr() :: nominal | opaque | type.

type_name()

(not exported)

-type type_name() :: atom().

type_specifier()

(not exported)

-type type_specifier() :: type() | signedness() | endianness() | unit().

type_specifier_list()

(not exported)

-type type_specifier_list() :: default | [type_specifier(), ...].

unary_op()

(not exported)

-type unary_op() :: '+' | '-' | 'bnot' | 'not'.

unit()

(not exported)

-type unit() :: {unit, 1..256}.

Functions

abstract(Data)

-spec abstract(Data) -> AbsTerm when Data :: term(), AbsTerm :: abstract_expr().

Converts the Erlang data structure Data into an abstract form of type AbsTerm. This function is the inverse of normalise/1.

erl_parse:abstract(T) is equivalent to erl_parse:abstract(T, 0).

abstract(Data, Options)

(since OTP R16B01)

-spec abstract(Data, Options) -> AbsTerm
                  when
                      Data :: term(),
                      Options :: Location | [Option],
                      Option :: {encoding, Encoding} | {line, Line} | {location, Location},
                      Encoding :: latin1 | unicode | utf8 | none | encoding_func(),
                      Line :: erl_anno:line(),
                      Location :: erl_anno:location(),
                      AbsTerm :: abstract_expr().

Converts the Erlang data structure Data into an abstract form of type AbsTerm.

Each node of AbsTerm is assigned an annotation, see erl_anno. The annotation contains the location given by option location or by option line. Option location overrides option line. If neither option location nor option line is given, 0 is used as location.

Option Encoding is used for selecting which integer lists to be considered as strings. The default is to use the encoding returned by function epp:default_encoding/0. Value none means that no integer lists are considered as strings. encoding_func() is called with one integer of a list at a time; if it returns true for every integer, the list is considered a string.

anno_from_term(Term)

(since OTP 18.0)

-spec anno_from_term(Term) -> erl_parse_tree() | form_info() when Term :: term().

Assumes that Term is a term with the same structure as a erl_parse tree, but with terms, say T, where a erl_parse tree has collections of annotations.

Returns a erl_parse tree where each term T is replaced by the value returned by erl_anno:from_term(T). The term Term is traversed in a depth-first, left-to-right fashion.

anno_to_term(Abstr)

(since OTP 18.0)

-spec anno_to_term(Abstr) -> term() when Abstr :: erl_parse_tree() | form_info().

Returns a term where each collection of annotations Anno of the nodes of the erl_parse tree Abstr is replaced by the term returned by erl_anno:to_term(Anno). The erl_parse tree is traversed in a depth-first, left-to-right fashion.

fold_anno(Fun, Acc0, Abstr)

(since OTP 18.0)

-spec fold_anno(Fun, Acc0, Abstr) -> Acc1
                   when
                       Fun :: fun((Anno, AccIn) -> AccOut),
                       Anno :: erl_anno:anno(),
                       Acc0 :: term(),
                       Acc1 :: term(),
                       AccIn :: term(),
                       AccOut :: term(),
                       Abstr :: erl_parse_tree() | form_info().

Updates an accumulator by applying Fun on each collection of annotations of the erl_parse tree Abstr.

The first call to Fun has AccIn as argument, the returned accumulator AccOut is passed to the next call, and so on. The final value of the accumulator is returned. The erl_parse tree is traversed in a depth-first, left-to-right fashion.

format_error(Message)

-spec format_error(any()) -> [char() | list()].

Uses an ErrorDescriptor and returns a string that describes the error.

This function is usually called implicitly when an ErrorInfo structure is processed (see section Error Information).

map_anno(Fun, Abstr)

(since OTP 18.0)

-spec map_anno(Fun, Abstr) -> NewAbstr
                  when
                      Fun :: fun((Anno) -> NewAnno),
                      Anno :: erl_anno:anno(),
                      NewAnno :: erl_anno:anno(),
                      Abstr :: erl_parse_tree() | form_info(),
                      NewAbstr :: erl_parse_tree() | form_info().

Modifies the erl_parse tree Abstr by applying Fun on each collection of annotations of the nodes of the erl_parse tree. The erl_parse tree is traversed in a depth-first, left-to-right fashion.

mapfold_anno(Fun, Acc0, Abstr)

(since OTP 18.0)

-spec mapfold_anno(Fun, Acc0, Abstr) -> {NewAbstr, Acc1}
                      when
                          Fun :: fun((Anno, AccIn) -> {NewAnno, AccOut}),
                          Anno :: erl_anno:anno(),
                          NewAnno :: erl_anno:anno(),
                          Acc0 :: term(),
                          Acc1 :: term(),
                          AccIn :: term(),
                          AccOut :: term(),
                          Abstr :: erl_parse_tree() | form_info(),
                          NewAbstr :: erl_parse_tree() | form_info().

Modifies the erl_parse tree Abstr by applying Fun on each collection of annotations of the nodes of the erl_parse tree, while at the same time updating an accumulator.

The first call to Fun has AccIn as second argument, the returned accumulator AccOut is passed to the next call, and so on. The modified erl_parse tree and the final value of the accumulator are returned. The erl_parse tree is traversed in a depth-first, left-to-right fashion.

new_anno(Term)

(since OTP 18.0)

-spec new_anno(Term) -> Abstr when Term :: term(), Abstr :: erl_parse_tree() | form_info().

Assumes that Term is a term with the same structure as a erl_parse tree, but with locations where a erl_parse tree has collections of annotations.

Returns a erl_parse tree where each location L is replaced by the value returned by erl_anno:new(L). The term Term is traversed in a depth-first, left-to-right fashion.

normalise(AbsTerm)

-spec normalise(AbsTerm) -> Data when AbsTerm :: abstract_expr(), Data :: term().

Converts the abstract form AbsTerm of a term into a conventional Erlang data structure (that is, the term itself). This function is the inverse of abstract/1.

parse_exprs(Tokens)

-spec parse_exprs(Tokens) -> {ok, ExprList} | {error, ErrorInfo}
                     when Tokens :: [token()], ExprList :: [abstract_expr()], ErrorInfo :: error_info().

Parses Tokens as if it was a list of expressions.

Returns one of the following:

{ok, ExprList} - The parsing was successful. ExprList is a list of the abstract forms of the parsed expressions.

{error, ErrorInfo} - An error occurred.

parse_form(Tokens)

-spec parse_form(Tokens) -> {ok, AbsForm} | {error, ErrorInfo}
                    when Tokens :: [token()], AbsForm :: abstract_form(), ErrorInfo :: error_info().

Parses Tokens as if it was a form.

Returns one of the following:

{ok, AbsForm} - The parsing was successful. AbsForm is the abstract form of the parsed form.

{error, ErrorInfo} - An error occurred.

parse_term(Tokens)

-spec parse_term(Tokens) -> {ok, Term} | {error, ErrorInfo}
                    when Tokens :: [token()], Term :: term(), ErrorInfo :: error_info().

Parses Tokens as if it was a term.

Returns one of the following:

{ok, Term} - The parsing was successful. Term is the Erlang term corresponding to the token list.

{error, ErrorInfo} - An error occurred.

tokens(AbsTerm)

-spec tokens(AbsTerm) -> Tokens when AbsTerm :: abstract_expr(), Tokens :: [token()].

Equivalent to tokens(AbsTerm, []).

tokens(AbsTerm, MoreTokens)

-spec tokens(AbsTerm, MoreTokens) -> Tokens
                when AbsTerm :: abstract_expr(), MoreTokens :: [token()], Tokens :: [token()].

Generates a list of tokens representing the abstract form AbsTerm of an expression. Optionally, MoreTokens is appended.

erl_pp

The Erlang pretty printer.

The functions in this module are used to generate aesthetically attractive representations of abstract forms, which are suitable for printing. All functions return (possibly deep) lists of characters and generate an error if the form is wrong.

All functions can have an optional argument, which specifies a hook that is called if an attempt is made to print an unknown form.

Note that if the functions in this module are used to convert abstract code back to Erlang source code, the enclosing function should first be processed by legalize_vars/1 in order to ensure that the output is semantically equivalent to the abstract code.

Known Limitations

It is not possible to have hook functions for unknown forms at other places than expressions.

See Also

erl_eval, erl_parse, io

Summary

Types

hook_function()

Optional argument HookFunction, shown in the functions described in this module, defines a function that is called when an unknown form occurs where there is to be a valid expression. If HookFunction is equal to none, there is no hook function.

option()

The option quote_singleton_atom_types is used to add quotes to all singleton atom types.

options()

Functions

attribute(Attribute)

Equivalent to attribute(Attribute, none).

attribute(Attribute, Options)

Same as form/1,2, but only for attribute Attribute.

expr(Expression)

Equivalent to expr(Expression, none).

expr(Expression, Options)

Equivalent to expr(Expression, 0, Options).

expr(Expression, Indent, Options)

Equivalent to expr(Expression, Indent, 0, Options).

expr(Expression, Indent, Precedence, Options)

Prints one expression.

exprs(Expressions)

Equivalent to exprs(Expressions, none).

exprs(Expressions, Options)

Equivalent to exprs(Expressions, 0, Options).

exprs(Expressions, Indent, Options)

Same as form/1,2, but only for the sequence of expressions in Expressions.

form(Form)

Equivalent to form(Form, none).

form(Form, Options)

Pretty prints a Form, which is an abstract form of a type that is returned by erl_parse:parse_form/1.

function(Function)

Equivalent to function(Function, none).

function(Function, Options)

Same as form/1,2, but only for function Function.

guard(Guard)

Equivalent to guard(Guard, none).

guard(Guard, Options)

Same as form/1,2, but only for the guard test Guard.

legalize_vars(Function)

The Erlang compiler will, when expanding records to tuples, introduce new variables in the abstract representation. As the expansion is done on the abstract representation, the compiler can safely name the new variables with names that are not syntactically valid in Erlang source code (the name starts with a lowercase letter), thus ensuring the uniqueness of the new names.

Types

hook_function()

(not exported)

-type hook_function() ::
          none |
          fun((Expr :: erl_parse:abstract_expr(),
               CurrentIndentation :: integer(),
               CurrentPrecedence :: non_neg_integer(),
               Options :: options()) ->
                  io_lib:chars()).

Optional argument HookFunction, shown in the functions described in this module, defines a function that is called when an unknown form occurs where there is to be a valid expression. If HookFunction is equal to none, there is no hook function.

The called hook function is to return a (possibly deep) list of characters. Function expr/4 is useful in a hook.

If CurrentIndentation is negative, there are no line breaks and only a space is used as a separator.

option()

(not exported)

-type option() ::
          {hook, hook_function()} |
          {encoding, latin1 | unicode | utf8} |
          {quote_singleton_atom_types, boolean()} |
          {linewidth, pos_integer()} |
          {indent, pos_integer()}.

The option quote_singleton_atom_types is used to add quotes to all singleton atom types.

The option linewidth controls the maximum line width for formatted lines (defaults to 72 characters).

The option indent controls the indention for formatted lines (defaults to 4 spaces).

options()

(not exported)

-type options() :: hook_function() | [option()].

Functions

attribute(Attribute)

-spec attribute(Attribute) -> io_lib:chars() when Attribute :: erl_parse:abstract_form().

Equivalent to attribute(Attribute, none).

attribute(Attribute, Options)

-spec attribute(Attribute, Options) -> io_lib:chars()
                   when Attribute :: erl_parse:abstract_form(), Options :: options().

Same as form/1,2, but only for attribute Attribute.

expr(Expression)

-spec expr(Expression) -> io_lib:chars() when Expression :: erl_parse:abstract_expr().

Equivalent to expr(Expression, none).

expr(Expression, Options)

-spec expr(Expression, Options) -> io_lib:chars()
              when Expression :: erl_parse:abstract_expr(), Options :: options().

Equivalent to expr(Expression, 0, Options).

expr(Expression, Indent, Options)

-spec expr(Expression, Indent, Options) -> io_lib:chars()
              when Expression :: erl_parse:abstract_expr(), Indent :: integer(), Options :: options().

Equivalent to expr(Expression, Indent, 0, Options).

expr(Expression, Indent, Precedence, Options)

-spec expr(Expression, Indent, Precedence, Options) -> io_lib:chars()
              when
                  Expression :: erl_parse:abstract_expr(),
                  Indent :: integer(),
                  Precedence :: non_neg_integer(),
                  Options :: options().

Prints one expression.

It is useful for implementing hooks (see section Known Limitations).

exprs(Expressions)

-spec exprs(Expressions) -> io_lib:chars() when Expressions :: [erl_parse:abstract_expr()].

Equivalent to exprs(Expressions, none).

exprs(Expressions, Options)

-spec exprs(Expressions, Options) -> io_lib:chars()
               when Expressions :: [erl_parse:abstract_expr()], Options :: options().

Equivalent to exprs(Expressions, 0, Options).

exprs(Expressions, Indent, Options)

-spec exprs(Expressions, Indent, Options) -> io_lib:chars()
               when
                   Expressions :: [erl_parse:abstract_expr()], Indent :: integer(), Options :: options().

Same as form/1,2, but only for the sequence of expressions in Expressions.

form(Form)

-spec form(Form) -> io_lib:chars() when Form :: erl_parse:abstract_form() | erl_parse:form_info().

Equivalent to form(Form, none).

form(Form, Options)

-spec form(Form, Options) -> io_lib:chars()
              when Form :: erl_parse:abstract_form() | erl_parse:form_info(), Options :: options().

Pretty prints a Form, which is an abstract form of a type that is returned by erl_parse:parse_form/1.

function(Function)

-spec function(Function) -> io_lib:chars() when Function :: erl_parse:abstract_form().

Equivalent to function(Function, none).

function(Function, Options)

-spec function(Function, Options) -> io_lib:chars()
                  when Function :: erl_parse:abstract_form(), Options :: options().

Same as form/1,2, but only for function Function.

guard(Guard)

-spec guard(Guard) -> io_lib:chars() when Guard :: [erl_parse:abstract_expr()].

Equivalent to guard(Guard, none).

guard(Guard, Options)

-spec guard(Guard, Options) -> io_lib:chars()
               when Guard :: [erl_parse:abstract_expr()], Options :: options().

Same as form/1,2, but only for the guard test Guard.

legalize_vars(Function)

(since OTP 25.0)

-spec legalize_vars(Function) -> erl_parse:abstract_form() when Function :: erl_parse:abstract_form().

The Erlang compiler will, when expanding records to tuples, introduce new variables in the abstract representation. As the expansion is done on the abstract representation, the compiler can safely name the new variables with names that are not syntactically valid in Erlang source code (the name starts with a lowercase letter), thus ensuring the uniqueness of the new names.

The above strategy leads to problems if a user wants to convert the abstract representation, using the functions of this module back to Erlang source code. Typically, pattern variables are output as atoms thus changing the sematics of the program. To solve this problem legalize_vars/1, when run on the abstract representation of a function, will return an equivalent function where all variables will have syntactically valid names.

erl_scan

The Erlang token scanner.

This module contains functions for tokenizing (scanning) characters into Erlang tokens.

Error Information

ErrorInfo is the standard ErrorInfo structure that is returned from all I/O modules. The format is as follows:

{ErrorLocation, Module, ErrorDescriptor}

A string describing the error is obtained with the following call:

Module:format_error(ErrorDescriptor)

Notes

The continuation of the first call to the re-entrant input functions must be []. For a complete description of how the re-entrant input scheme works, see Armstrong, Virding and Williams: 'Concurrent Programming in Erlang', Chapter 13.

See Also

erl_anno, erl_parse, io

Summary

Types

category()

char_spec()

error_description()

error_info()

option()

options()

resword_fun()

return_cont()

symbol()

text_fun()

token()

tokens()

tokens_result()

Functions

category(Token)

Returns the category of Token.

column(Token)

Returns the column of Token's collection of annotations.

end_location(Token)

Returns the end location of the text of Token's collection of annotations. If there is no text, undefined is returned.

format_error(ErrorDescriptor)

Uses an ErrorDescriptor and returns a string that describes the error or warning. This function is usually called implicitly when an ErrorInfo structure is processed (see section Error Information).

line(Token)

Returns the line of Token's collection of annotations.

location(Token)

Returns the location of Token's collection of annotations.

reserved_word(Atom)

Returns true if Atom is an Erlang reserved word, otherwise false.

string(String)

Equivalent to string(String, 1).

string(String, StartLocation)

Equivalent to string(String, StartLocation, []).

string(String, StartLocation, Options)

Takes the list of characters String and tries to scan (tokenize) them.

symbol(Token)

Returns the symbol of Token.

text(Token)

Returns the text of Token's collection of annotations. If there is no text, undefined is returned.

tokens(Continuation, CharSpec, StartLocation)

Equivalent to tokens(Continuation, CharSpec, StartLocation, []).

tokens(Continuation, CharSpec, StartLocation, Options)

This is the re-entrant scanner, which scans characters until either a dot ('.' followed by a white space) or eof is reached.

Types

category()

(not exported)

-type category() :: atom().

char_spec()

(not exported)

-type char_spec() :: string() | eof.

error_description()

(not exported)

-type error_description() :: term().

error_info()

-type error_info() :: {erl_anno:location(), module(), error_description()}.

option()

(not exported)

-type option() ::
          return | return_white_spaces | return_comments | text |
          {reserved_word_fun, resword_fun()} |
          {text_fun, text_fun()} |
          {compiler_internal, [term()]}.

options()

-type options() :: option() | [option()].

resword_fun()

(not exported)

-type resword_fun() :: fun((atom()) -> boolean()).

return_cont()

-opaque return_cont()

symbol()

(not exported)

-type symbol() :: atom() | float() | integer() | string().

text_fun()

(not exported)

-type text_fun() :: fun((atom(), string()) -> boolean()).

token()

-type token() :: {category(), Anno :: erl_anno:anno(), symbol()} | {category(), Anno :: erl_anno:anno()}.

tokens()

-type tokens() :: [token()].

tokens_result()

-type tokens_result() ::
          {ok, Tokens :: tokens(), EndLocation :: erl_anno:location()} |
          {eof, EndLocation :: erl_anno:location()} |
          {error, ErrorInfo :: error_info(), EndLocation :: erl_anno:location()}.

Functions

category(Token)

(since OTP 18.0)

-spec category(Token) -> category() when Token :: token().

Returns the category of Token.

column(Token)

(since OTP 18.0)

-spec column(Token) -> erl_anno:column() | undefined when Token :: token().

Returns the column of Token's collection of annotations.

end_location(Token)

(since OTP 18.0)

-spec end_location(Token) -> erl_anno:location() | undefined when Token :: token().

Returns the end location of the text of Token's collection of annotations. If there is no text, undefined is returned.

format_error(ErrorDescriptor)

-spec format_error(ErrorDescriptor) -> string() when ErrorDescriptor :: error_description().

Uses an ErrorDescriptor and returns a string that describes the error or warning. This function is usually called implicitly when an ErrorInfo structure is processed (see section Error Information).

line(Token)

(since OTP 18.0)

-spec line(Token) -> erl_anno:line() when Token :: token().

Returns the line of Token's collection of annotations.

location(Token)

(since OTP 18.0)

-spec location(Token) -> erl_anno:location() when Token :: token().

Returns the location of Token's collection of annotations.

reserved_word(Atom)

-spec reserved_word(Atom :: atom()) -> boolean().

Returns true if Atom is an Erlang reserved word, otherwise false.

string(String)

-spec string(String) -> Return
                when
                    String :: string(),
                    Return ::
                        {ok, Tokens :: tokens(), EndLocation} |
                        {error, ErrorInfo :: error_info(), ErrorLocation},
                    EndLocation :: erl_anno:location(),
                    ErrorLocation :: erl_anno:location().

Equivalent to string(String, 1).

string(String, StartLocation)

-spec string(String, StartLocation) -> Return
                when
                    String :: string(),
                    Return ::
                        {ok, Tokens :: tokens(), EndLocation} |
                        {error, ErrorInfo :: error_info(), ErrorLocation},
                    StartLocation :: erl_anno:location(),
                    EndLocation :: erl_anno:location(),
                    ErrorLocation :: erl_anno:location().

Equivalent to string(String, StartLocation, []).

string(String, StartLocation, Options)

-spec string(String, StartLocation, Options) -> Return
                when
                    String :: string(),
                    Options :: options(),
                    Return ::
                        {ok, Tokens :: tokens(), EndLocation} |
                        {error, ErrorInfo :: error_info(), ErrorLocation},
                    StartLocation :: erl_anno:location(),
                    EndLocation :: erl_anno:location(),
                    ErrorLocation :: erl_anno:location().

Takes the list of characters String and tries to scan (tokenize) them.

Returns one of the following:

{ok, Tokens, EndLocation} - Tokens are the Erlang tokens from String. EndLocation is the first location after the last token.

{error, ErrorInfo, ErrorLocation} - An error occurred. ErrorLocation is the first location after the erroneous token.

StartLocation indicates the initial location when scanning starts. If StartLocation is a line, Anno, EndLocation, and ErrorLocation are lines. If StartLocation is a pair of a line and a column, Anno takes the form of an opaque compound data type, and EndLocation and ErrorLocation are pairs of a line and a column. The token annotations contain information about the column and the line where the token begins, as well as the text of the token (if option text is specified), all of which can be accessed by calling column/1, line/1, location/1, and text/1.

A token is a tuple containing information about syntactic category, the token annotations, and the terminal symbol. For punctuation characters (such as ; and |) and reserved words, the category and the symbol coincide, and the token is represented by a two-tuple. Three-tuples have one of the following forms:

{atom, Anno, atom()}

{char, Anno, char()}

{comment, Anno, string()}

{float, Anno, float()}

{integer, Anno, integer()}

{var, Anno, atom()}

{white_space, Anno, string()}

Valid options:

{reserved_word_fun, reserved_word_fun()} - A callback function that is called when the scanner has found an unquoted atom. If the function returns true, the unquoted atom itself becomes the category of the token. If the function returns false, atom becomes the category of the unquoted atom.

return_comments - Return comment tokens.

return_white_spaces - Return white space tokens. By convention, a newline character, if present, is always the first character of the text (there cannot be more than one newline in a white space token).

return - Short for [return_comments, return_white_spaces].

text - Include the token text in the token annotation. The text is the part of the input corresponding to the token. See also text_fun.

{text_fun, text_fun()} - A callback function used to determine whether the full text for the token shall be included in the token annotation. Arguments of the function are the category of the token and the full token string. This is only used when text is not present. If neither are present the text will not be saved in the token annotation.

{compiler_internal, term()} - Pass compiler-internal options to the scanner. The set of internal options understood by the scanner should be considered experimental and can thus be changed at any time without prior warning.

The following options are currently understood:

ssa_checks - Tokenizes source code annotations used for encoding tests on the BEAM SSA code produced by the compiler.

symbol(Token)

(since OTP 18.0)

-spec symbol(Token) -> symbol() when Token :: token().

Returns the symbol of Token.

text(Token)

(since OTP 18.0)

-spec text(Token) -> erl_anno:text() | undefined when Token :: token().

Returns the text of Token's collection of annotations. If there is no text, undefined is returned.

tokens(Continuation, CharSpec, StartLocation)

-spec tokens(Continuation, CharSpec, StartLocation) -> Return
                when
                    Continuation :: return_cont() | [],
                    CharSpec :: char_spec(),
                    StartLocation :: erl_anno:location(),
                    Return ::
                        {done, Result :: tokens_result(), LeftOverChars :: char_spec()} |
                        {more, Continuation1 :: return_cont()}.

Equivalent to tokens(Continuation, CharSpec, StartLocation, []).

tokens(Continuation, CharSpec, StartLocation, Options)

-spec tokens(Continuation, CharSpec, StartLocation, Options) -> Return
                when
                    Continuation :: return_cont() | [],
                    CharSpec :: char_spec(),
                    StartLocation :: erl_anno:location(),
                    Options :: options(),
                    Return ::
                        {done, Result :: tokens_result(), LeftOverChars :: char_spec()} |
                        {more, Continuation1 :: return_cont()}.

This is the re-entrant scanner, which scans characters until either a dot ('.' followed by a white space) or eof is reached.

It returns:

{done, Result, LeftOverChars} - Indicates that there is sufficient input data to get a result. Result is:

{ok, Tokens, EndLocation} - The scanning was successful. Tokens is the list of tokens including dot.

{eof, EndLocation} - End of file was encountered before any more tokens.

{error, ErrorInfo, EndLocation} - An error occurred. LeftOverChars is the remaining characters of the input data, starting from EndLocation.

{more, Continuation1} - More data is required for building a term. Continuation1 must be passed in a new call to tokens/3,4 when more data is available.

The CharSpec eof signals end of file. LeftOverChars then takes the value eof as well.

For a description of the options, see string/3.

ms_transform

A parse transformation that translates fun syntax into match specifications.

This module provides the parse transformation that makes calls to ets and dbg:fun2ms/1 translate into literal match specifications. It also provides the back end for the same functions when called from the Erlang shell.

The translation from funs to match specifications is accessed through the two "pseudo functions" ets:fun2ms/1 and dbg:fun2ms/1.

As everyone trying to use ets:select/2 or dbg seems to end up reading this manual page, this description is an introduction to the concept of match specifications.

Read the whole manual page if it is the first time you are using the transformations.

Match specifications are used more or less as filters. They resemble usual Erlang matching in a list comprehension or in a fun used with lists:foldl/3, and so on. However, the syntax of pure match specifications is awkward, as they are made up purely by Erlang terms, and the language has no syntax to make the match specifications more readable.

As the execution and structure of the match specifications are like that of a fun, it is more straightforward to write it using the familiar fun syntax and to have that translated into a match specification automatically. A real fun is clearly more powerful than the match specifications allow, but bearing the match specifications in mind, and what they can do, it is still more convenient to write it all as a fun. This module contains the code that translates the fun syntax into match specification terms.

Example 1

Using ets:select/2 and a match specification, one can filter out rows of a table and construct a list of tuples containing relevant parts of the data in these rows. One can use ets:foldl/3 instead, but the ets:select/2 call is far more efficient. Without the translation provided by ms_transform, one must struggle with writing match specifications terms to accommodate this.

Consider a simple table of employees:

-record(emp, {empno,     %Employee number as a string, the key
              surname,   %Surname of the employee
              givenname, %Given name of employee
              dept,      %Department, one of {dev,sales,prod,adm}
              empyear}). %Year the employee was employed

We create the table using:

ets:new(emp_tab, [{keypos,#emp.empno},named_table,ordered_set]).

We fill the table with randomly chosen data:

[{emp,"011103","Black","Alfred",sales,2000},
 {emp,"041231","Doe","John",prod,2001},
 {emp,"052341","Smith","John",dev,1997},
 {emp,"076324","Smith","Ella",sales,1995},
 {emp,"122334","Weston","Anna",prod,2002},
 {emp,"535216","Chalker","Samuel",adm,1998},
 {emp,"789789","Harrysson","Joe",adm,1996},
 {emp,"963721","Scott","Juliana",dev,2003},
 {emp,"989891","Brown","Gabriel",prod,1999}]

Assuming that we want the employee numbers of everyone in the sales department, there are several ways.

ets:match/2 can be used:

1> ets:match(emp_tab, {'_', '$1', '_', '_', sales, '_'}).
[["011103"],["076324"]]

ets:match/2 uses a simpler type of match specification, but it is still unreadable, and one has little control over the returned result. It is always a list of lists.

ets:foldl/3 or ets:foldr/3 can be used to avoid the nested lists:

ets:foldr(fun(#emp{empno = E, dept = sales},Acc) -> [E | Acc];
             (_,Acc) -> Acc
          end,
          [],
          emp_tab).

The result is ["011103","076324"]. The fun is straightforward, so the only problem is that all the data from the table must be transferred from the table to the calling process for filtering. That is inefficient compared to the ets:match/2 call where the filtering can be done "inside" the emulator and only the result is transferred to the process.

Consider a "pure" ets:select/2 call that does what ets:foldr does:

ets:select(emp_tab, [{#emp{empno = '$1', dept = sales, _='_'},[],['$1']}]).

Although the record syntax is used, it is still hard to read and even harder to write. The first element of the tuple, #emp{empno = '$1', dept = sales, _='_'}, tells what to match. Elements not matching this are not returned, as in the ets:match/2 example. The second element, the empty list, is a list of guard expressions, which we do not need. The third element is the list of expressions constructing the return value (in ETS this is almost always a list containing one single term). In our case '$1' is bound to the employee number in the head (first element of the tuple), and hence the employee number is returned. The result is ["011103","076324"], as in the ets:foldr/3 example, but the result is retrieved much more efficiently in terms of execution speed and memory consumption.

Using ets:fun2ms/1, we can combine the ease of use of the ets:foldr/3 and the efficiency of the pure ets:select/2 example:

-include_lib("stdlib/include/ms_transform.hrl").

ets:select(emp_tab, ets:fun2ms(
                      fun(#emp{empno = E, dept = sales}) ->
                              E
                      end)).

This example requires no special knowledge of match specifications to understand. The head of the fun matches what you want to filter out and the body returns what you want returned. As long as the fun can be kept within the limits of the match specifications, there is no need to transfer all table data to the process for filtering as in the ets:foldr/3 example. It is easier to read than the ets:foldr/3 example, as the select call in itself discards anything that does not match, while the fun of the ets:foldr/3 call needs to handle both the elements matching and the ones not matching.

In the ets:fun2ms/1 example above, it is needed to include ms_transform.hrl in the source code, as this is what triggers the parse transformation of the ets:fun2ms/1 call to a valid match specification. This also implies that the transformation is done at compile time (except when called from the shell) and therefore takes no resources in runtime. That is, although you use the more intuitive fun syntax, it gets as efficient in runtime as writing match specifications by hand.

Example 2

Assume that we want to get all the employee numbers of employees hired before year 2000. Using ets:match/2 is not an alternative here, as relational operators cannot be expressed there. Once again, ets:foldr/3 can do it (slowly, but correct):

ets:foldr(fun(#emp{empno = E, empyear = Y},Acc) when Y < 2000 -> [E | Acc];
                  (_,Acc) -> Acc
          end,
          [],
          emp_tab).

The result is ["052341","076324","535216","789789","989891"], as expected. The equivalent expression using a handwritten match specification would look like this:

ets:select(emp_tab, [{#emp{empno = '$1', empyear = '$2', _='_'},
                     [{'<', '$2', 2000}],
                     ['$1']}]).

This gives the same result. [{'<', '$2', 2000}] is in the guard part and therefore discards anything that does not have an empyear (bound to '$2' in the head) less than 2000, as the guard in the foldr/3 example.

We write it using ets:fun2ms/1:

-include_lib("stdlib/include/ms_transform.hrl").

ets:select(emp_tab, ets:fun2ms(
                      fun(#emp{empno = E, empyear = Y}) when Y < 2000 ->
                           E
                      end)).

Example 3

Assume that we want the whole object matching instead of only one element. One alternative is to assign a variable to every part of the record and build it up once again in the body of the fun, but the following is easier:

ets:select(emp_tab, ets:fun2ms(
                      fun(Obj = #emp{empno = E, empyear = Y})
                         when Y < 2000 ->
                              Obj
                      end)).

As in ordinary Erlang matching, you can bind a variable to the whole matched object using a "match inside the match", that is, a =. Unfortunately in funs translated to match specifications, it is allowed only at the "top-level", that is, matching the whole object arriving to be matched into a separate variable. If you are used to writing match specifications by hand, we mention that variable A is simply translated into '$_'. Alternatively, pseudo function object/0 also returns the whole matched object, see section Warnings and Restrictions.

Example 4

This example concerns the body of the fun. Assume that all employee numbers beginning with zero (0) must be changed to begin with one (1) instead, and that we want to create the list [{<Old empno>,<New empno>}]:

ets:select(emp_tab, ets:fun2ms(
                      fun(#emp{empno = [$0 | Rest] }) ->
                              {[$0|Rest],[$1|Rest]}
                      end)).

This query hits the feature of partially bound keys in table type ordered_set, so that not the whole table needs to be searched, only the part containing keys beginning with 0 is looked into.

Example 5

The fun can have many clauses. Assume that we want to do the following:

If an employee started before 1997, return the tuple {inventory, <employee number>}.

If an employee started 1997 or later, but before 2001, return {rookie, <employee number>}.

For all other employees, return {newbie, <employee number>}, except for those named Smith as they would be affronted by anything other than the tag guru and that is also what is returned for their numbers: {guru, <employee number>}.

This is accomplished as follows:

ets:select(emp_tab, ets:fun2ms(
                      fun(#emp{empno = E, surname = "Smith" }) ->
                              {guru,E};
                         (#emp{empno = E, empyear = Y}) when Y < 1997  ->
                              {inventory, E};
                         (#emp{empno = E, empyear = Y}) when Y > 2001  ->
                              {newbie, E};
                         (#emp{empno = E, empyear = Y}) -> % 1997 -- 2001
                              {rookie, E}
                      end)).

The result is as follows:

[{rookie,"011103"},
 {rookie,"041231"},
 {guru,"052341"},
 {guru,"076324"},
 {newbie,"122334"},
 {rookie,"535216"},
 {inventory,"789789"},
 {newbie,"963721"},
 {rookie,"989891"}]

Useful BIFs

What more can you do? A simple answer is: see the documentation of match specifications in ERTS User's Guide. However, the following is a brief overview of the most useful "built-in functions" that you can use when the fun is to be translated into a match specification by ets:fun2ms/1. It is not possible to call other functions than those allowed in match specifications. No "usual" Erlang code can be executed by the fun that is translated by ets:fun2ms/1. The fun is limited exactly to the power of the match specifications, which is unfortunate, but the price one must pay for the execution speed of ets:select/2 compared to ets:foldl/foldr.

The head of the fun is a head matching (or mismatching) one parameter, one object of the table we select from. The object is always a single variable (can be _) or a tuple, as ETS, Dets, and Mnesia tables include that. The match specification returned by ets:fun2ms/1 can be used with dets:select/2 and mnesia:select/2, and with ets:select/2. The use of = in the head is allowed (and encouraged) at the top-level.

The guard section can contain any guard expression of Erlang. The following is a list of BIFs and expressions:

Type tests: is_atom, is_float, is_integer, is_list, is_number, is_pid, is_port, is_reference, is_tuple, is_binary, is_function, is_record

Boolean operators: not, and, or, andalso, orelse

Relational operators: >, >=, <, =<, =:=, ==, =/=, /=

Arithmetic: +, -, *, div, rem

Bitwise operators: band, bor, bxor, bnot, bsl, bsr

The guard BIFs: abs, element, hd, length, node, round, size, byte_size, tl, trunc, binary_part, self

Contrary to the fact with "handwritten" match specifications, the is_record guard works as in ordinary Erlang code.

Semicolons (;) in guards are allowed, the result is (as expected) one "match specification clause" for each semicolon-separated part of the guard. The semantics is identical to the Erlang semantics.

The body of the fun is used to construct the resulting value. When selecting from tables, one usually construct a suiting term here, using ordinary Erlang term construction, like tuple parentheses, list brackets, and variables matched out in the head, possibly with the occasional constant. Whatever expressions are allowed in guards are also allowed here, but no special functions exist except object and bindings (see further down), which returns the whole matched object and all known variable bindings, respectively.

The dbg variants of match specifications have an imperative approach to the match specification body, the ETS dialect has not. The fun body for ets:fun2ms/1 returns the result without side effects. As matching (=) in the body of the match specifications is not allowed (for performance reasons) the only thing left, more or less, is term construction.

Example with dbg

This section describes the slightly different match specifications translated by dbg:fun2ms/1.

The same reasons for using the parse transformation apply to dbg, maybe even more, as filtering using Erlang code is not a good idea when tracing (except afterwards, if you trace to file). The concept is similar to that of ets:fun2ms/1 except that you usually use it directly from the shell (which can also be done with ets:fun2ms/1).

The following is an example module to trace on:

-module(toy).

-export([start/1, store/2, retrieve/1]).

start(Args) ->
    toy_table = ets:new(toy_table, Args).

store(Key, Value) ->
    ets:insert(toy_table, {Key,Value}).

retrieve(Key) ->
    [{Key, Value}] = ets:lookup(toy_table, Key),
    Value.

During model testing, the first test results in {badmatch,16} in {toy,start,1}, why?

We suspect the ets:new/2 call, as we match hard on the return value, but want only the particular new/2 call with toy_table as first parameter. So we start a default tracer on the node:

1> dbg:tracer().
{ok,<0.88.0>}

We turn on call tracing for all processes, we want to make a pretty restrictive trace pattern, so there is no need to call trace only a few processes (usually it is not):

2> dbg:p(all,call).
{ok,[{matched,nonode@nohost,25}]}

We specify the filter, we want to view calls that resemble ets:new(toy_table, <something>):

3> dbg:tp(ets,new,dbg:fun2ms(fun([toy_table,_]) -> true end)).
{ok,[{matched,nonode@nohost,1},{saved,1}]}

As can be seen, the fun used with dbg:fun2ms/1 takes a single list as parameter instead of a single tuple. The list matches a list of the parameters to the traced function. A single variable can also be used. The body of the fun expresses, in a more imperative way, actions to be taken if the fun head (and the guards) matches. true is returned here, only because the body of a fun cannot be empty. The return value is discarded.

The following trace output is received during test:

(<0.86.0>) call ets:new(toy_table, [ordered_set])

Assume that we have not found the problem yet, and want to see what ets:new/2 returns. We use a slightly different trace pattern:

4> dbg:tp(ets,new,dbg:fun2ms(fun([toy_table,_]) -> return_trace() end)).

The following trace output is received during test:

(<0.86.0>) call ets:new(toy_table,[ordered_set])
(<0.86.0>) returned from ets:new/2 -> 24

The call to return_trace results in a trace message when the function returns. It applies only to the specific function call triggering the match specification (and matching the head/guards of the match specification). This is by far the most common call in the body of a dbg match specification.

The test now fails with {badmatch,24} because the atom toy_table does not match the number returned for an unnamed table. So, the problem is found, the table is to be named, and the arguments supplied by the test program do not include named_table. We rewrite the start function:

start(Args) ->
    toy_table = ets:new(toy_table, [named_table|Args]).

With the same tracing turned on, the following trace output is received:

(<0.86.0>) call ets:new(toy_table,[named_table,ordered_set])
(<0.86.0>) returned from ets:new/2 -> toy_table

Assume that the module now passes all testing and goes into the system. After a while, it is found that table toy_table grows while the system is running and that there are many elements with atoms as keys. We expected only integer keys and so does the rest of the system, but clearly not the entire system. We turn on call tracing and try to see calls to the module with an atom as the key:

1> dbg:tracer().
{ok,<0.88.0>}
2> dbg:p(all,call).
{ok,[{matched,nonode@nohost,25}]}
3> dbg:tpl(toy,store,dbg:fun2ms(fun([A,_]) when is_atom(A) -> true end)).
{ok,[{matched,nonode@nohost,1},{saved,1}]}

We use dbg:tpl/3 to ensure to catch local calls (assume that the module has grown since the smaller version and we are unsure if this inserting of atoms is not done locally). When in doubt, always use local call tracing.

Assume that nothing happens when tracing in this way. The function is never called with these parameters. We conclude that someone else (some other module) is doing it and realize that we must trace on ets:insert/2 and want to see the calling function. The calling function can be retrieved using the match specification function caller. To get it into the trace message, the match specification function message must be used. The filter call looks like this (looking for calls to ets:insert/2):

4> dbg:tpl(ets,insert,dbg:fun2ms(fun([toy_table,{A,_}]) when is_atom(A) ->
                                    message(caller())
                                  end)).
{ok,[{matched,nonode@nohost,1},{saved,2}]}

The caller is now displayed in the "additional message" part of the trace output, and the following is displayed after a while:

(<0.86.0>) call ets:insert(toy_table,{garbage,can}) ({evil_mod,evil_fun,2})

You have realized that function evil_fun of the evil_mod module, with arity 2, is causing all this trouble.

This example illustrates the most used calls in match specifications for dbg. The other, more esoteric, calls are listed and explained in Match specifications in Erlang in ERTS User's Guide, as they are beyond the scope of this description.

Warnings and Restrictions

The following warnings and restrictions apply to the funs used in with ets:fun2ms/1 and dbg:fun2ms/1.

Warning

To use the pseudo functions triggering the translation, ensure to include the header file ms_transform.hrl in the source code. Failure to do so possibly results in runtime errors rather than compile time, as the expression can be valid as a plain Erlang program without translation.

Warning

The fun must be literally constructed inside the parameter list to the pseudo functions. The fun cannot be bound to a variable first and then passed to ets:fun2ms/1 or dbg:fun2ms/1. For example, ets:fun2ms(fun(A) -> A end) works, but not F = fun(A) -> A end, ets:fun2ms(F). The latter results in a compile-time error if the header is included, otherwise a runtime error.

Many restrictions apply to the fun that is translated into a match specification. To put it simple: you cannot use anything in the fun that you cannot use in a match specification. This means that, among others, the following restrictions apply to the fun itself:

Functions written in Erlang cannot be called, neither can local functions, global functions, or real funs.

Everything that is written as a function call is translated into a match specification call to a built-in function, so that the call is_list(X) is translated to {'is_list', '$1'} ('$1' is only an example, the numbering can vary). If one tries to call a function that is not a match specification built-in, it causes an error.

Variables occurring in the head of the fun are replaced by match specification variables in the order of occurrence, so that fragment fun({A,B,C}) is replaced by {'$1', '$2', '$3'}, and so on. Every occurrence of such a variable in the match specification is replaced by a match specification variable in the same way, so that the fun fun({A,B}) when is_atom(A) -> B end is translated into [{{'$1','$2'},[{is_atom,'$1'}],['$2']}].

Variables that are not included in the head are imported from the environment and made into match specification const expressions. Example from the shell:

1> X = 25.
25
2> ets:fun2ms(fun({A,B}) when A > X -> B end).
[{{'$1','$2'},[{'>','$1',{const,25}}],['$2']}]

Matching with = cannot be used in the body. It can only be used on the top-level in the head of the fun. Example from the shell again:

1> ets:fun2ms(fun({A,[B|C]} = D) when A > B -> D end).
[{{'$1',['$2'|'$3']},[{'>','$1','$2'}],['$_']}]
2> ets:fun2ms(fun({A,[B|C]=D}) when A > B -> D end).
Error: fun with head matching ('=' in head) cannot be translated into
match_spec
{error,transform_error}
3> ets:fun2ms(fun({A,[B|C]}) when A > B -> D = [B|C], D end).
Error: fun with body matching ('=' in body) is illegal as match_spec
{error,transform_error}

All variables are bound in the head of a match specification, so the translator cannot allow multiple bindings. The special case when matching is done on the top-level makes the variable bind to '$_' in the resulting match specification. It is to allow a more natural access to the whole matched object. Pseudo function object() can be used instead, see below.

The following expressions are translated equally:

ets:fun2ms(fun({a,_} = A) -> A end).
ets:fun2ms(fun({a,_}) -> object() end).

The special match specification variables '$_' and '$*' can be accessed through the pseudo functions object() (for '$_') and bindings() (for '$*'). As an example, one can translate the following ets:match_object/2 call to a ets:select/2 call:

ets:match_object(Table, {'$1',test,'$2'}).

This is the same as:

ets:select(Table, ets:fun2ms(fun({A,test,B}) -> object() end)).

In this simple case, the former expression is probably preferable in terms of readability.

The ets:select/2 call conceptually looks like this in the resulting code:

ets:select(Table, [{{'$1',test,'$2'},[],['$_']}]).

Matching on the top-level of the fun head can be a more natural way to access '$_', see above.

Term constructions/literals are translated as much as is needed to get them into valid match specification. This way tuples are made into match specification tuple constructions (a one element tuple containing the tuple) and constant expressions are used when importing variables from the environment. Records are also translated into plain tuple constructions, calls to element, and so on. The guard test is_record/2 is translated into match specification code using the three parameter version that is built into match specification, so that is_record(A,t) is translated into {is_record,'$1',t,5} if the record size of record type t is 5.

Language constructions such as case, if, and catch that are not present in match specifications are not allowed.

If header file ms_transform.hrl is not included, the fun is not translated, which can result in a runtime error (depending on whether the fun is valid in a pure Erlang context).

Ensure that the header is included when using ets and dbg:fun2ms/1 in compiled code.

If pseudo function triggering the translation is ets:fun2ms/1, the head of the fun must contain a single variable or a single tuple. If the pseudo function is dbg:fun2ms/1, the head of the fun must contain a single variable or a single list.

The translation from funs to match specifications is done at compile time, so runtime performance is not affected by using these pseudo functions.

For more information about match specifications, see the Match specifications in Erlang in ERTS User's Guide.

Summary

Functions

format_error(Error)

Takes an error code returned by one of the other functions in the module and creates a textual description of the error.

parse_transform(Forms, Options)

Implements the transformation at compile time. This function is called by the compiler to do the source code transformation if and when header file ms_transform.hrl is included in the source code.

transform_from_shell(Dialect, Clauses, BoundEnvironment)

Implements the transformation when the fun2ms/1 functions are called from the shell. In this case, the abstract form is for one single fun (parsed by the Erlang shell). All imported variables are to be in the key-value list passed as BoundEnvironment. The result is a term, normalized, that is, not in abstract format.

Functions

format_error(Error)

-spec format_error(Error) -> Chars when Error :: {error, module(), term()}, Chars :: io_lib:chars().

Takes an error code returned by one of the other functions in the module and creates a textual description of the error.

parse_transform(Forms, Options)

-spec parse_transform(Forms, Options) -> Forms2 | Errors | Warnings
                         when
                             Forms :: [erl_parse:abstract_form() | erl_parse:form_info()],
                             Forms2 :: [erl_parse:abstract_form() | erl_parse:form_info()],
                             Options :: term(),
                             Errors :: {error, ErrInfo :: [tuple()], WarnInfo :: []},
                             Warnings :: {warning, Forms2, WarnInfo :: [tuple()]}.

Implements the transformation at compile time. This function is called by the compiler to do the source code transformation if and when header file ms_transform.hrl is included in the source code.

For information about how to use this parse transformation, see ets and dbg:fun2ms/1.

For a description of match specifications, see section Match Specification in Erlang in ERTS User's Guide.

transform_from_shell(Dialect, Clauses, BoundEnvironment)

-spec transform_from_shell(Dialect, Clauses, BoundEnvironment) -> term()
                              when
                                  Dialect :: ets | dbg,
                                  Clauses :: [erl_parse:abstract_clause()],
                                  BoundEnvironment :: erl_eval:binding_struct().

Implements the transformation when the fun2ms/1 functions are called from the shell. In this case, the abstract form is for one single fun (parsed by the Erlang shell). All imported variables are to be in the key-value list passed as BoundEnvironment. The result is a term, normalized, that is, not in abstract format.

array

Functional, extendible arrays.

Arrays can have fixed size, or can grow automatically as needed. A default value is used for entries that have not been explicitly set.

Arrays uses zero-based indexing. This is a deliberate design choice and differs from other Erlang data structures, for example, tuples.

Unless specified by the user when the array is created, the default value is the atom undefined. There is no difference between an unset entry and an entry that has been explicitly set to the same value as the default one (compare reset/2). If you need to differentiate between unset and set entries, ensure that the default value cannot be confused with the values of set entries.

The array never shrinks automatically. If an index I has been used to set an entry successfully, all indices in the range [0,I] stay accessible unless the array size is explicitly changed by calling resize/2.

Examples:

Create a fixed-size array with entries 0-9 set to undefined:

A0 = array:new(10).
10 = array:size(A0).

Create an extendible array and set entry 17 to true, causing the array to grow automatically:

A1 = array:set(17, true, array:new()).
18 = array:size(A1).

Read back a stored value:

true = array:get(17, A1).

Accessing an unset entry returns default value:

undefined = array:get(3, A1)

Accessing an entry beyond the last set entry also returns the default value, if the array does not have fixed size:

undefined = array:get(18, A1).

"Sparse" functions ignore default-valued entries:

A2 = array:set(4, false, A1).
[{4, false}, {17, true}] = array:sparse_to_orddict(A2).

An extendible array can be made fixed-size later:

A3 = array:fix(A2).

A fixed-size array does not grow automatically and does not allow accesses beyond the last set entry:

{'EXIT',{badarg,_}} = (catch array:set(18, true, A3)).
{'EXIT',{badarg,_}} = (catch array:get(18, A3)).

Summary

Types

array()

array(Type)

A functional, extendible array. The representation is not documented and is subject to change without notice. Notice that arrays cannot be directly compared for equality.

array_indx()

array_opt()

array_opts()

indx_pair(Type)

indx_pairs(Type)

Functions

default(Array)

Gets the value used for uninitialized entries.

fix(Array)

Fixes the array size. This prevents it from growing automatically upon insertion.

foldl(Function, InitialAcc, Array)

Folds the array elements using the specified function and initial accumulator value. The elements are visited in order from the lowest index to the highest.

foldr(Function, InitialAcc, Array)

Folds the array elements right-to-left using the specified function and initial accumulator value. The elements are visited in order from the highest index to the lowest.

from(Function, State)

Equivalent to from(Fun, State, undefined).

from(Function, State, Default)

Creates an extendible array with values obtained with Function(State).

from_list(List)

Equivalent to from_list(List, undefined).

from_list(List, Default)

Converts a list to an extendible array. Default is used as the value for uninitialized entries of the array.

from_orddict(Orddict)

Equivalent to from_orddict(Orddict, undefined).

from_orddict(Orddict, Default)

Converts an ordered list of pairs {Index, Value} to a corresponding extendible array. Default is used as the value for uninitialized entries of the array.

get(I, Array)

Gets the value of entry I.

is_array(X)

Returns true if X is an array, otherwise false.

is_fix(Array)

Checks if the array has fixed size. Returns true if the array is fixed, otherwise false.

map(Function, Array)

Maps the specified function onto each array element. The elements are visited in order from the lowest index to the highest.

new()

Creates a new, extendible array with initial size zero.

new(Options)

Creates a new array according to the specified options. By default, the array is extendible and has initial size zero. Array indices start at 0.

new(Size, Options)

Creates a new array according to the specified size and options.

relax(Array)

Makes the array resizable. (Reverses the effects of fix/1.)

reset(I, Array)

Resets entry I to the default value for the array. If the value of entry I is the default value, the array is returned unchanged.

resize(Array)

Changes the array size to that reported by sparse_size/1. If the specified array has fixed size, also the resulting array has fixed size.

resize(Size, Array)

Change the array size.

set(I, Value, Array)

Sets entry I of the array to Value.

size(Array)

Gets the number of entries in the array. Entries are numbered from 0 to size(Array)-1. Hence, this is also the index of the first entry that is guaranteed to not have been previously set.

sparse_foldl(Function, InitialAcc, Array)

Folds the array elements using the specified function and initial accumulator value, skipping default-valued entries. The elements are visited in order from the lowest index to the highest.

sparse_foldr(Function, InitialAcc, Array)

Folds the array elements right-to-left using the specified function and initial accumulator value, skipping default-valued entries. The elements are visited in order from the highest index to the lowest.

sparse_map(Function, Array)

Maps the specified function onto each array element, skipping default-valued entries. The elements are visited in order from the lowest index to the highest.

sparse_size(Array)

Gets the number of entries in the array up until the last non-default-valued entry. That is, returns I+1 if I is the last non-default-valued entry in the array, or zero if no such entry exists.

sparse_to_list(Array)

Converts the array to a list, skipping default-valued entries.

sparse_to_orddict(Array)

Converts the array to an ordered list of pairs {Index, Value}, skipping default-valued entries.

to_list(Array)

Converts the array to a list.

to_orddict(Array)

Converts the array to an ordered list of pairs {Index, Value}.

Types

array()

-type array() :: array(term()).

array(Type)

-opaque array(Type)

A functional, extendible array. The representation is not documented and is subject to change without notice. Notice that arrays cannot be directly compared for equality.

array_indx()

(not exported)

-type array_indx() :: non_neg_integer().

array_opt()

(not exported)

-type array_opt() ::
          {fixed, boolean()} |
          fixed |
          {default, Type :: term()} |
          {size, N :: non_neg_integer()} |
          (N :: non_neg_integer()).

array_opts()

(not exported)

-type array_opts() :: array_opt() | [array_opt()].

indx_pair(Type)

(not exported)

-type indx_pair(Type) :: {Index :: array_indx(), Type}.

indx_pairs(Type)

(not exported)

-type indx_pairs(Type) :: [indx_pair(Type)].

Functions

default(Array)

-spec default(Array :: array(Type)) -> Value :: Type.

Gets the value used for uninitialized entries.

See also new/2.

fix(Array)

-spec fix(Array :: array(Type)) -> array(Type).

Fixes the array size. This prevents it from growing automatically upon insertion.

See also set/3 and relax/1.

foldl(Function, InitialAcc, Array)

-spec foldl(Function, InitialAcc :: A, Array :: array(Type)) -> B
               when Function :: fun((Index :: array_indx(), Value :: Type, Acc :: A) -> B).

Folds the array elements using the specified function and initial accumulator value. The elements are visited in order from the lowest index to the highest.

If Function is not a function, the call fails with reason badarg.

See also foldr/3, map/2, sparse_foldl/3.

foldr(Function, InitialAcc, Array)

-spec foldr(Function, InitialAcc :: A, Array :: array(Type)) -> B
               when Function :: fun((Index :: array_indx(), Value :: Type, Acc :: A) -> B).

Folds the array elements right-to-left using the specified function and initial accumulator value. The elements are visited in order from the highest index to the lowest.

If Function is not a function, the call fails with reason badarg.

See also foldl/3, map/2.

from(Function, State)

(since OTP 29.0)

-spec from(Function, State :: term()) -> array(Type)
              when Function :: fun((State0 :: term()) -> {Type, State1 :: term()} | done).

Equivalent to from(Fun, State, undefined).

from(Function, State, Default)

(since OTP 29.0)

-spec from(Function, State :: term(), Default :: term()) -> array(Type)
              when Function :: fun((State0 :: term()) -> {Type, State1 :: term()} | done).

Creates an extendible array with values obtained with Function(State).

The 'Function(State)' shall return {Value, NewState} or done, and is invoked until done is returned, otherwise the call fails with reason badarg.

Default is used as the value for uninitialized entries of the array.

Examples

1> Floats = << <<N:32/float-native>> || N <- lists:seq(0, 2047)>>.
2> BinToVal = fun(I) ->
     case Floats of
         <<_:I/binary, N:32/float-native, _/binary>> ->
             {N, I+4};
         _ ->
             done
     end
   end.
3> A = array:from(BinToVal, 0).
4> array:get(10, A).
10.0
5> array:size(A).
2048
6> ValToBin = fun(_K, V, Acc) -> <<Acc/binary, V:32/float-native>> end.
7> Floats == array:foldl(ValToBin, <<>>, A).
true

See also new/2, from_list/1, foldl/3.

from_list(List)

-spec from_list(List :: [Value :: Type]) -> array(Type).

Equivalent to from_list(List, undefined).

from_list(List, Default)

-spec from_list(List :: [Value :: Type], Default :: term()) -> array(Type).

Converts a list to an extendible array. Default is used as the value for uninitialized entries of the array.

If List is not a proper list, the call fails with reason badarg.

See also new/2, to_list/1.

from_orddict(Orddict)

-spec from_orddict(Orddict :: indx_pairs(Value :: Type)) -> array(Type).

Equivalent to from_orddict(Orddict, undefined).

from_orddict(Orddict, Default)

-spec from_orddict(Orddict :: indx_pairs(Value :: Type), Default :: Type) -> array(Type).

Converts an ordered list of pairs {Index, Value} to a corresponding extendible array. Default is used as the value for uninitialized entries of the array.

If Orddict is not a proper, ordered list of pairs whose first elements are non-negative integers, the call fails with reason badarg.

See also new/2, to_orddict/1.

get(I, Array)

-spec get(I :: array_indx(), Array :: array(Type)) -> Value :: Type.

Gets the value of entry I.

If I is not a non-negative integer, or if the array has fixed size and I is larger than the maximum index, the call fails with reason badarg.

If the array does not have fixed size, the default value for any index I greater than size(Array)-1 is returned.

See also set/3.

is_array(X)

-spec is_array(X :: term()) -> boolean().

Returns true if X is an array, otherwise false.

Notice that the check is only shallow, as there is no guarantee that X is a well-formed array representation even if this function returns true.

is_fix(Array)

-spec is_fix(Array :: array()) -> boolean().

Checks if the array has fixed size. Returns true if the array is fixed, otherwise false.

See also fix/1.

map(Function, Array)

-spec map(Function, Array :: array(Type1)) -> array(Type2)
             when Function :: fun((Index :: array_indx(), Type1) -> Type2).

Maps the specified function onto each array element. The elements are visited in order from the lowest index to the highest.

If Function is not a function, the call fails with reason badarg.

See also foldl/3, foldr/3, sparse_map/2.

new()

-spec new() -> array().

Creates a new, extendible array with initial size zero.

See also new/1, new/2.

new(Options)

-spec new(Options :: array_opts()) -> array().

Creates a new array according to the specified options. By default, the array is extendible and has initial size zero. Array indices start at 0.

Options is a single term or a list of terms, selected from the following:

N::integer() >= 0 or {size, N::integer() >= 0} - Specifies the initial array size; this also implies {fixed, true}. If N is not a non-negative integer, the call fails with reason badarg.

fixed or {fixed, true} - Creates a fixed-size array. See also fix/1.

{fixed, false} - Creates an extendible (non-fixed-size) array.

{default, Value} - Sets the default value for the array to Value.

Options are processed in the order they occur in the list, that is, later options have higher precedence.

The default value is used as the value of uninitialized entries, and cannot be changed once the array has been created.

Examples:

array:new(100)

creates a fixed-size array of size 100.

array:new({default,0})

creates an empty, extendible array whose default value is 0.

array:new([{size,10},{fixed,false},{default,-1}])

creates an extendible array with initial size 10 whose default value is -1.

See also fix/1, from_list/2, get/2, new/0, new/2, set/3.

new(Size, Options)

-spec new(Size :: non_neg_integer(), Options :: array_opts()) -> array().

Creates a new array according to the specified size and options.

If Size is not a non-negative integer, the call fails with reason badarg. By default, the array has fixed size. Notice that any size specifications in Options override parameter Size.

If Options is a list, this is equivalent to new([{size, Size} | Options]), otherwise it is equivalent to new([{size, Size} | [Options]]). However, using this function directly is more efficient.

Example:

array:new(100, {default,0})

creates a fixed-size array of size 100, whose default value is 0.

See also new/1.

relax(Array)

-spec relax(Array :: array(Type)) -> array(Type).

Makes the array resizable. (Reverses the effects of fix/1.)

See also fix/1.

reset(I, Array)

-spec reset(I :: array_indx(), Array :: array(Type)) -> array(Type).

Resets entry I to the default value for the array. If the value of entry I is the default value, the array is returned unchanged.

Reset never changes the array size. Shrinking can be done explicitly by calling resize/2.

If I is not a non-negative integer, or if the array has fixed size and I is larger than the maximum index, the call fails with reason badarg; compare set/3

See also new/2, set/3.

resize(Array)

-spec resize(Array :: array(Type)) -> array(Type).

Changes the array size to that reported by sparse_size/1. If the specified array has fixed size, also the resulting array has fixed size.

See also resize/2, sparse_size/1.

resize(Size, Array)

-spec resize(Size :: non_neg_integer(), Array :: array(Type)) -> array(Type).

Change the array size.

If Size is not a non-negative integer, the call fails with reason badarg. If the specified array has fixed size, also the resulting array has fixed size.

set(I, Value, Array)

-spec set(I :: array_indx(), Value :: Type, Array :: array(Type)) -> array(Type).

Sets entry I of the array to Value.

If I is not a non-negative integer, or if the array has fixed size and I is larger than the maximum index, the call fails with reason badarg.

If the array does not have fixed size, and I is greater than size(Array)-1, the array grows to size I+1.

See also get/2, reset/2.

size(Array)

-spec size(Array :: array()) -> non_neg_integer().

Gets the number of entries in the array. Entries are numbered from 0 to size(Array)-1. Hence, this is also the index of the first entry that is guaranteed to not have been previously set.

See also set/3, sparse_size/1.

sparse_foldl(Function, InitialAcc, Array)

-spec sparse_foldl(Function, InitialAcc :: A, Array :: array(Type)) -> B
                      when Function :: fun((Index :: array_indx(), Value :: Type, Acc :: A) -> B).

Folds the array elements using the specified function and initial accumulator value, skipping default-valued entries. The elements are visited in order from the lowest index to the highest.

If Function is not a function, the call fails with reason badarg.

See also foldl/3, sparse_foldr/3.

sparse_foldr(Function, InitialAcc, Array)

-spec sparse_foldr(Function, InitialAcc :: A, Array :: array(Type)) -> B
                      when Function :: fun((Index :: array_indx(), Value :: Type, Acc :: A) -> B).

Folds the array elements right-to-left using the specified function and initial accumulator value, skipping default-valued entries. The elements are visited in order from the highest index to the lowest.

If Function is not a function, the call fails with reason badarg.

See also foldr/3, sparse_foldl/3.

sparse_map(Function, Array)

-spec sparse_map(Function, Array :: array(Type1)) -> array(Type2)
                    when Function :: fun((Index :: array_indx(), Type1) -> Type2).

Maps the specified function onto each array element, skipping default-valued entries. The elements are visited in order from the lowest index to the highest.

If Function is not a function, the call fails with reason badarg.

See also map/2.

sparse_size(Array)

-spec sparse_size(Array :: array()) -> non_neg_integer().

Gets the number of entries in the array up until the last non-default-valued entry. That is, returns I+1 if I is the last non-default-valued entry in the array, or zero if no such entry exists.

See also resize/1, size/1.

sparse_to_list(Array)

-spec sparse_to_list(Array :: array(Type)) -> [Value :: Type].

Converts the array to a list, skipping default-valued entries.

See also to_list/1.

sparse_to_orddict(Array)

-spec sparse_to_orddict(Array :: array(Type)) -> indx_pairs(Value :: Type).

Converts the array to an ordered list of pairs {Index, Value}, skipping default-valued entries.

See also to_orddict/1.

to_list(Array)

-spec to_list(Array :: array(Type)) -> [Value :: Type].

Converts the array to a list.

See also from_list/2, sparse_to_list/1.

to_orddict(Array)

-spec to_orddict(Array :: array(Type)) -> indx_pairs(Value :: Type).

Converts the array to an ordered list of pairs {Index, Value}.

See also from_orddict/2, sparse_to_orddict/1.

dets

A disk-based term storage.

This module provides a term storage on file. The stored terms, in this module called objects, are tuples such that one element is defined to be the key. A Dets table is a collection of objects with the key at the same position stored on a file.

This module is used by the Mnesia application, and is provided "as is" for users who are interested in efficient storage of Erlang terms on disk only. Many applications only need to store some terms in a file. Mnesia adds transactions, queries, and distribution. The size of Dets files cannot exceed 2 GB. If larger tables are needed, table fragmentation in Mnesia can be used.

Three types of Dets tables exist:

set. A table of this type has at most one object with a given key. If an object with a key already present in the table is inserted, the existing object is overwritten by the new object.

bag. A table of this type has zero or more different objects with a given key.

duplicate_bag. A table of this type has zero or more possibly matching objects with a given key.

Dets tables must be opened before they can be updated or read, and when finished they must be properly closed. If a table is not properly closed, Dets automatically repairs the table. This can take a substantial time if the table is large. A Dets table is closed when the process which opened the table terminates. If many Erlang processes (users) open the same Dets table, they share the table. The table is properly closed when all users have either terminated or closed the table. Dets tables are not properly closed if the Erlang runtime system terminates abnormally.

Note

A ^C command abnormally terminates an Erlang runtime system in a Unix environment with a break-handler.

As all operations performed by Dets are disk operations, it is important to realize that a single look-up operation involves a series of disk seek and read operations. The Dets functions are therefore much slower than the corresponding ets functions, although Dets exports a similar interface.

Dets organizes data as a linear hash list and the hash list grows gracefully as more data is inserted into the table. Space management on the file is performed by what is called a buddy system. The current implementation keeps the entire buddy system in RAM, which implies that if the table gets heavily fragmented, quite some memory can be used up. The only way to defragment a table is to close it and then open it again with option repair set to force.

Notice that type ordered_set in Ets is not yet provided by Dets, neither is the limited support for concurrent updates that makes a sequence of first and next calls safe to use on fixed ETS tables. Both these features may be provided by Dets in a future release of Erlang/OTP. Until then, the Mnesia application (or some user-implemented method for locking) must be used to implement safe concurrency. Currently, no Erlang/OTP library has support for ordered disk-based term storage.

All Dets functions return {error, Reason} if an error occurs (first/1 and next/2 are exceptions, they exit the process with the error tuple). If badly formed arguments are specified, all functions exit the process with a badarg message.

See Also

ets, mnesia, qlc

Summary

Types

access()

auto_save()

bindings_cont()

Opaque continuation used by match/1 and match/3.

cont()

Opaque continuation used by bchunk/2.

keypos()

match_spec()

Match specifications, see section Match Specification in Erlang in ERTS User's Guide and the ms_transform module.

no_slots()

object()

object_cont()

Opaque continuation used by match_object/1 and match_object/3.

pattern()

For a description of patterns, see ets:match/2.

select_cont()

Opaque continuation used by select/1 and select/3.

tab_name()

type()

Functions

all()

Returns a list of the names of all open tables on this node.

bchunk(Name, Continuation)

Returns a list of objects stored in a table. The exact representation of the returned objects is not public.

close(Name)

Closes a table. Only processes that have opened a table are allowed to close it.

delete(Name, Key)

Deletes all objects with key Key from table Name.

delete_all_objects(Name)

Deletes all objects from a table in almost constant time. However, if the table if fixed, delete_all_objects(T) is equivalent to match_delete(T, '_').

delete_object(Name, Object)

Deletes all instances of a specified object from a table. If a table is of type bag or duplicate_bag, this function can be used to delete only some of the objects with a specified key.

first(Name)

Returns the first key stored in table Name according to the internal order of the table, or '$end_of_table' if the table is empty.

foldl(Function, Acc0, Name)

Equivalent to foldr/3.

foldr(Function, Acc0, Name)

Calls Function on successive elements of table Name together with an extra argument AccIn. The table elements are traversed in unspecified order. Function must return a new accumulator that is passed to the next call. Acc0 is returned if the table is empty.

from_ets(Name, EtsTab)

Deletes all objects of table Name and then inserts all the objects of the ETS table EtsTab. The objects are inserted in unspecified order. As ets:safe_fixtable/2 is called, the ETS table must be public or owned by the calling process.

info(Name)

Returns information about table Name as a list of tuples

info(Name, Item)

Returns the information associated with Item for table Name. In addition to the {Item, Value} pairs defined for info/1, the following items are allowed

init_table(Name, InitFun)

Equivalent to init_table(Name, InitFun, []).

init_table(Name, InitFun, Options)

Replaces the existing objects of table Name with objects created by calling the input function InitFun.

insert(Name, Objects)

Inserts one or more objects into the table Name. If there already exists an object with a key matching the key of some of the given objects and the table type is set, the old object will be replaced.

insert_new(Name, Objects)

Inserts one or more objects into table Name. If there already exists some object with a key matching the key of any of the specified objects, the table is not updated and false is returned. Otherwise the objects are inserted and true returned.

is_compatible_bchunk_format(Name, BchunkFormat)

Returns true if it would be possible to initialize table Name, using init_table/3 with option {format, bchunk}, with objects read with bchunk/2 from some table T, such that calling info(T, bchunk_format) returns BchunkFormat.

is_dets_file(Filename)

Returns true if file Filename is a Dets table, otherwise false.

lookup(Name, Key)

Returns a list of all objects with key Key stored in table Name, for example

match(Continuation)

Matches some objects stored in a table and returns a non-empty list of the bindings matching a specified pattern in some unspecified order. The table, the pattern, and the number of objects that are matched are all defined by Continuation, which has been returned by a previous call to match/1 or match/3.

match(Name, Pattern)

Returns for each object of table Name that matches Pattern a list of bindings in some unspecified order. For a description of patterns, see ets:match/2. If the keypos'th element of Pattern is unbound, all table objects are matched. If the keypos'th element is bound, only the objects with the correct key are matched.

match(Name, Pattern, N)

Matches some or all objects of table Name and returns a non-empty list of the bindings that match Pattern in some unspecified order. For a description of patterns, see ets:match/2.

match_delete(Name, Pattern)

Deletes all objects that match Pattern from table Name. For a description of patterns, see ets:match/2.

match_object(Continuation)

Returns a non-empty list of some objects stored in a table that match a given pattern in some unspecified order. The table, the pattern, and the number of objects that are matched are all defined by Continuation, which has been returned by a previous call to match_object/1 or match_object/3.

match_object(Name, Pattern)

Returns a list of all objects of table Name that match Pattern in some unspecified order. For a description of patterns, see ets:match/2.

match_object(Name, Pattern, N)

Matches some or all objects stored in table Name and returns a non-empty list of the objects that match Pattern in some unspecified order. For a description of patterns, see ets:match/2.

member(Name, Key)

Works like lookup/2, but does not return the objects. Returns true if one or more table elements has key Key, otherwise false.

next(Name, Key1)

Returns either the key following Key1 in table Name according to the internal order of the table, or '$end_of_table' if there is no next key.

open_file(Filename)

Opens an existing table. If the table is not properly closed, it is repaired. The returned reference is to be used as the table name. This function is most useful for debugging purposes.

open_file(Name, Args)

Opens a table. An empty Dets table is created if no file exists.

pid2name(Pid)

Returns the table name given the pid of a process that handles requests to a table, or undefined if there is no such table.

repair_continuation(Continuation, MatchSpec)

This function can be used to restore an opaque continuation returned by select/3 or select/1 if the continuation has passed through external term format (been sent between nodes or stored on disk).

safe_fixtable(Name, Fix)

If Fix is true, table Name is fixed (once more) by the calling process, otherwise the table is released. The table is also released when a fixing process terminates.

select(Continuation)

Applies a match specification to some objects stored in a table and returns a non-empty list of the results. The table, the match specification, and the number of objects that are matched are all defined by Continuation, which is returned by a previous call to select/1 or select/3.

select(Name, MatchSpec)

Returns the results of applying match specification MatchSpec to all or some objects stored in table Name. The order of the objects is not specified. For a description of match specifications, see the ERTS User's Guide.

select(Name, MatchSpec, N)

Returns the results of applying match specification MatchSpec to some or all objects stored in table Name. The order of the objects is not specified. For a description of match specifications, see the ERTS User's Guide.

select_delete(Name, MatchSpec)

Deletes each object from table Name such that applying match specification MatchSpec to the object returns value true. For a description of match specifications, see the ERTS User's Guide. Returns the number of deleted objects.

slot(Name, I)

The objects of a table are distributed among slots, starting with slot 0 and ending with slot n. Returns the list of objects associated with slot I. If I > n, '$end_of_table' is returned.

sync(Name)

Ensures that all updates made to table Name are written to disk. This also applies to tables that have been opened with flag ram_file set to true. In this case, the contents of the RAM file are flushed to disk.

table(Name)

Equivalent to table(Name, []).

table(Name, Options)

Returns a Query List Comprehension (QLC) query handle. The qlc module provides a query language aimed mainly for Mnesia, but ETS tables, Dets tables, and lists are also recognized by qlc as sources of data. Calling dets:table/1,2 is the means to make Dets table Name usable to qlc.

to_ets(Name, EtsTab)

Inserts the objects of the Dets table Name into the ETS table EtsTab. The order in which the objects are inserted is not specified. The existing objects of the ETS table are kept unless overwritten.

traverse(Name, Fun)

Applies Fun to each object stored in table Name in some unspecified order. Different actions are taken depending on the return value of Fun. The following Fun return values are allowed

update_counter(Name, Key, Increment)

Updates the object with key Key stored in table Name of type set by adding Incr to the element at the Pos:th position. The new counter value is returned. If no position is specified, the element directly following the key is updated.

Types

access()

(not exported)

-type access() :: read | read_write.

auto_save()

(not exported)

-type auto_save() :: timeout().

bindings_cont()

-opaque bindings_cont()

Opaque continuation used by match/1 and match/3.

cont()

-opaque cont()

Opaque continuation used by bchunk/2.

keypos()

(not exported)

-type keypos() :: pos_integer().

match_spec()

(not exported)

-type match_spec() :: ets:match_spec().

Match specifications, see section Match Specification in Erlang in ERTS User's Guide and the ms_transform module.

no_slots()

(not exported)

-type no_slots() :: default | non_neg_integer().

object()

(not exported)

-type object() :: tuple().

object_cont()

-opaque object_cont()

Opaque continuation used by match_object/1 and match_object/3.

pattern()

(not exported)

-type pattern() :: atom() | tuple().

For a description of patterns, see ets:match/2.

select_cont()

-opaque select_cont()

Opaque continuation used by select/1 and select/3.

tab_name()

-type tab_name() :: term().

type()

(not exported)

-type type() :: bag | duplicate_bag | set.

Functions

all()

-spec all() -> [tab_name()].

Returns a list of the names of all open tables on this node.

bchunk(Name, Continuation)

-spec bchunk(Name, Continuation) -> {Continuation2, Data} | '$end_of_table' | {error, Reason}
                when
                    Name :: tab_name(),
                    Continuation :: start | cont(),
                    Continuation2 :: cont(),
                    Data :: binary() | tuple(),
                    Reason :: term().

Returns a list of objects stored in a table. The exact representation of the returned objects is not public.

The lists of data can be used for initializing a table by specifying value bchunk to option format of function init_table/3. The Mnesia application uses this function for copying open tables.

Unless the table is protected using safe_fixtable/2, calls to bchunk/2 do possibly not work as expected if concurrent updates are made to the table.

The first time bchunk/2 is called, an initial continuation, the atom start, must be provided.

bchunk/2 returns a tuple {Continuation2, Data}, where Data is a list of objects. Continuation2 is another continuation that is to be passed on to a subsequent call to bchunk/2. With a series of calls to bchunk/2, all table objects can be extracted.

bchunk/2 returns '$end_of_table' when all objects are returned, or {error, Reason} if an error occurs.

close(Name)

-spec close(Name) -> ok | {error, Reason} when Name :: tab_name(), Reason :: term().

Closes a table. Only processes that have opened a table are allowed to close it.

All open tables must be closed before the system is stopped. If an attempt is made to open a table that is not properly closed, Dets automatically tries to repair it.

delete(Name, Key)

-spec delete(Name, Key) -> ok | {error, Reason} when Name :: tab_name(), Key :: term(), Reason :: term().

Deletes all objects with key Key from table Name.

delete_all_objects(Name)

-spec delete_all_objects(Name) -> ok | {error, Reason} when Name :: tab_name(), Reason :: term().

Deletes all objects from a table in almost constant time. However, if the table if fixed, delete_all_objects(T) is equivalent to match_delete(T, '_').

delete_object(Name, Object)

-spec delete_object(Name, Object) -> ok | {error, Reason}
                       when Name :: tab_name(), Object :: object(), Reason :: term().

Deletes all instances of a specified object from a table. If a table is of type bag or duplicate_bag, this function can be used to delete only some of the objects with a specified key.

first(Name)

-spec first(Name) -> Key | '$end_of_table' when Name :: tab_name(), Key :: term().

Returns the first key stored in table Name according to the internal order of the table, or '$end_of_table' if the table is empty.

Unless the table is protected using safe_fixtable/2, subsequent calls to next/2 do possibly not work as expected if concurrent updates are made to the table.

If an error occurs, the process is exited with an error tuple {error, Reason}. The error tuple is not returned, as it cannot be distinguished from a key.

There are two reasons why first/1 and next/2 are not to be used: they are not efficient, and they prevent the use of key '$end_of_table', as this atom is used to indicate the end of the table. If possible, use functions match, match_object, and select for traversing tables.

foldl(Function, Acc0, Name)

-spec foldl(Function, Acc0, Name) -> Acc | {error, Reason}
               when
                   Name :: tab_name(),
                   Function :: fun((Object :: object(), AccIn) -> AccOut),
                   Acc0 :: term(),
                   Acc :: term(),
                   AccIn :: term(),
                   AccOut :: term(),
                   Reason :: term().

Equivalent to foldr/3.

foldr(Function, Acc0, Name)

-spec foldr(Function, Acc0, Name) -> Acc | {error, Reason}
               when
                   Name :: tab_name(),
                   Function :: fun((Object :: object(), AccIn) -> AccOut),
                   Acc0 :: term(),
                   Acc :: term(),
                   AccIn :: term(),
                   AccOut :: term(),
                   Reason :: term().

Calls Function on successive elements of table Name together with an extra argument AccIn. The table elements are traversed in unspecified order. Function must return a new accumulator that is passed to the next call. Acc0 is returned if the table is empty.

from_ets(Name, EtsTab)

-spec from_ets(Name, EtsTab) -> ok | {error, Reason}
                  when Name :: tab_name(), EtsTab :: ets:table(), Reason :: term().

Deletes all objects of table Name and then inserts all the objects of the ETS table EtsTab. The objects are inserted in unspecified order. As ets:safe_fixtable/2 is called, the ETS table must be public or owned by the calling process.

info(Name)

-spec info(Name) -> InfoList | undefined
              when
                  Name :: tab_name(),
                  InfoList :: [InfoTuple],
                  InfoTuple ::
                      {file_size, non_neg_integer()} |
                      {filename, file:name()} |
                      {keypos, keypos()} |
                      {size, non_neg_integer()} |
                      {type, type()}.

Returns information about table Name as a list of tuples:

{file_size, integer() >= 0}} - The file size, in bytes.

{filename, file:name/0 } - The name of the file where objects are stored.

{keypos, keypos/0 } - The key position.

{size, integer() >= 0} - The number of objects stored in the table.

{type, type/0 } - The table type.

info(Name, Item)

-spec info(Name, Item) -> Value | undefined
              when
                  Name :: tab_name(),
                  Item ::
                      access | auto_save | bchunk_format | hash | file_size | filename | keypos |
                      memory | no_keys | no_objects | no_slots | owner | ram_file | safe_fixed |
                      safe_fixed_monotonic_time | size | type,
                  Value :: term().

Returns the information associated with Item for table Name. In addition to the {Item, Value} pairs defined for info/1, the following items are allowed:

{access, access/0 } - The access mode.

{auto_save, auto_save/0 } - The autosave interval.

{bchunk_format, binary()} - An opaque binary describing the format of the objects returned by bchunk/2. The binary can be used as argument to is_compatible_bchunk_format/2.

{hash, Hash} - Describes which BIF is used to calculate the hash values of the objects stored in the Dets table. Possible values of Hash:

phash - Implies that the erlang:phash/2 BIF is used.

phash2 - Implies that the erlang:phash2/1 BIF is used.

{memory, integer() >= 0} - The file size, in bytes. The same value is associated with item file_size.

{no_keys, integer >= 0()} - The number of different keys stored in the table.

{no_objects, integer >= 0()} - The number of objects stored in the table.

{no_slots, {Min, Used, Max}} - The number of slots of the table. Min is the minimum number of slots, Used is the number of currently used slots, and Max is the maximum number of slots.

{owner, pid()} - The pid of the process that handles requests to the Dets table.

{ram_file, boolean()} - Whether the table is kept in RAM.

{safe_fixed_monotonic_time, SafeFixed} - If the table is fixed, SafeFixed is a tuple {FixedAtTime, [{Pid,RefCount}]}. FixedAtTime is the time when the table was first fixed, and Pid is the pid of the process that fixes the table RefCount times. There can be any number of processes in the list. If the table is not fixed, SafeFixed is the atom false.

FixedAtTime corresponds to the result returned by erlang:monotonic_time/0 at the time of fixation. The use of safe_fixed_monotonic_time is time warp safe.

{safe_fixed, SafeFixed} - The same as {safe_fixed_monotonic_time, SafeFixed} except the format and value of FixedAtTime.

FixedAtTime corresponds to the result returned by erlang:timestamp/0 at the time of fixation. Notice that when the system uses single or multi time warp modes, this can produce strange results. This is because the use of safe_fixed is not time warp safe. Time warp safe code must use safe_fixed_monotonic_time instead.

init_table(Name, InitFun)

-spec init_table(Name, InitFun) -> ok | {error, Reason}
                    when
                        Name :: tab_name(),
                        InitFun :: fun((Arg) -> Res),
                        Arg :: read | close,
                        Res :: end_of_input | {[object()], InitFun} | {Data, InitFun} | term(),
                        Reason :: term(),
                        Data :: binary() | tuple().

Equivalent to init_table(Name, InitFun, []).

init_table(Name, InitFun, Options)

-spec init_table(Name, InitFun, Options) -> ok | {error, Reason}
                    when
                        Name :: tab_name(),
                        InitFun :: fun((Arg) -> Res),
                        Arg :: read | close,
                        Res :: end_of_input | {[object()], InitFun} | {Data, InitFun} | term(),
                        Options :: Option | [Option],
                        Option :: {min_no_slots, no_slots()} | {format, term | bchunk},
                        Reason :: term(),
                        Data :: binary() | tuple().

Replaces the existing objects of table Name with objects created by calling the input function InitFun.

The reason for using this function rather than calling insert/2 is that of efficiency. Notice that the input functions are called by the process that handles requests to the Dets table, not by the calling process.

When called with argument read, function InitFun is assumed to return end_of_input when there is no more input, or {Objects, Fun}, where Objects is a list of objects and Fun is a new input function. Any other value Value is returned as an error {error, {init_fun, Value}}. Each input function is called exactly once, and if an error occurs, the last function is called with argument close, the reply of which is ignored.

If the table type is set and more than one object exists with a given key, one of the objects is chosen. This is not necessarily the last object with the given key in the sequence of objects returned by the input functions. Avoid duplicate keys, otherwise the file becomes unnecessarily fragmented. This holds also for duplicated objects stored in tables of type bag.

It is important that the table has a sufficient number of slots for the objects. If not, the hash list starts to grow when init_table/2 returns, which significantly slows down access to the table for a period of time. The minimum number of slots is set by the open_file/2 option min_no_slots and returned by the info/2 item no_slots. See also option min_no_slots below.

Argument Options is a list of {Key, Val} tuples, where the following values are allowed:

{min_no_slots, no_slots()} - Specifies the estimated number of different keys to be stored in the table. The open_file/2 option with the same name is ignored, unless the table is created, in which case performance can be enhanced by supplying an estimate when initializing the table.

{format, Format} - Specifies the format of the objects returned by function InitFun. If Format is term (the default), InitFun is assumed to return a list of tuples. If Format is bchunk, InitFun is assumed to return Data as returned by bchunk/2. This option overrides option min_no_slots.

insert(Name, Objects)

-spec insert(Name, Objects) -> ok | {error, Reason}
                when Name :: tab_name(), Objects :: object() | [object()], Reason :: term().

Inserts one or more objects into the table Name. If there already exists an object with a key matching the key of some of the given objects and the table type is set, the old object will be replaced.

insert_new(Name, Objects)

-spec insert_new(Name, Objects) -> boolean() | {error, Reason}
                    when Name :: tab_name(), Objects :: object() | [object()], Reason :: term().

Inserts one or more objects into table Name. If there already exists some object with a key matching the key of any of the specified objects, the table is not updated and false is returned. Otherwise the objects are inserted and true returned.

is_compatible_bchunk_format(Name, BchunkFormat)

-spec is_compatible_bchunk_format(Name, BchunkFormat) -> boolean()
                                     when Name :: tab_name(), BchunkFormat :: binary().

Returns true if it would be possible to initialize table Name, using init_table/3 with option {format, bchunk}, with objects read with bchunk/2 from some table T, such that calling info(T, bchunk_format) returns BchunkFormat.

is_dets_file(Filename)

-spec is_dets_file(Filename) -> boolean() | {error, Reason}
                      when Filename :: file:name(), Reason :: term().

Returns true if file Filename is a Dets table, otherwise false.

lookup(Name, Key)

-spec lookup(Name, Key) -> Objects | {error, Reason}
                when Name :: tab_name(), Key :: term(), Objects :: [object()], Reason :: term().

Returns a list of all objects with key Key stored in table Name, for example:

2> dets:open_file(abc, [{type, bag}]).
{ok,abc}
3> dets:insert(abc, {1,2,3}).
ok
4> dets:insert(abc, {1,3,4}).
ok
5> dets:lookup(abc, 1).
[{1,2,3},{1,3,4}]

If the table type is set, the function returns either the empty list or a list with one object, as there cannot be more than one object with a given key. If the table type is bag or duplicate_bag, the function returns a list of arbitrary length.

Notice that the order of objects returned is unspecified. In particular, the order in which objects were inserted is not reflected.

match(Continuation)

-spec match(Continuation) -> {[Match], Continuation2} | '$end_of_table' | {error, Reason}
               when
                   Continuation :: bindings_cont(),
                   Continuation2 :: bindings_cont(),
                   Match :: [term()],
                   Reason :: term().

Matches some objects stored in a table and returns a non-empty list of the bindings matching a specified pattern in some unspecified order. The table, the pattern, and the number of objects that are matched are all defined by Continuation, which has been returned by a previous call to match/1 or match/3.

When all table objects are matched, '$end_of_table' is returned.

match(Name, Pattern)

-spec match(Name, Pattern) -> [Match] | {error, Reason}
               when Name :: tab_name(), Pattern :: pattern(), Match :: [term()], Reason :: term().

Returns for each object of table Name that matches Pattern a list of bindings in some unspecified order. For a description of patterns, see ets:match/2. If the keypos'th element of Pattern is unbound, all table objects are matched. If the keypos'th element is bound, only the objects with the correct key are matched.

match(Name, Pattern, N)

-spec match(Name, Pattern, N) -> {[Match], Continuation} | '$end_of_table' | {error, Reason}
               when
                   Name :: tab_name(),
                   Pattern :: pattern(),
                   N :: default | non_neg_integer(),
                   Continuation :: bindings_cont(),
                   Match :: [term()],
                   Reason :: term().

Matches some or all objects of table Name and returns a non-empty list of the bindings that match Pattern in some unspecified order. For a description of patterns, see ets:match/2.

A tuple of the bindings and a continuation is returned, unless the table is empty, in which case '$end_of_table' is returned. The continuation is to be used when matching further objects by calling match/1.

If the keypos'th element of Pattern is bound, all table objects are matched. If the keypos'th element is unbound, all table objects are matched, N objects at a time, until at least one object matches or the end of the table is reached. The default, indicated by giving N the value default, is to let the number of objects vary depending on the sizes of the objects. All objects with the same key are always matched at the same time, which implies that more than N objects can sometimes be matched.

The table is always to be protected using safe_fixtable/2 before calling match/3, otherwise errors can occur when calling match/1.

match_delete(Name, Pattern)

-spec match_delete(Name, Pattern) -> ok | {error, Reason}
                      when Name :: tab_name(), Pattern :: pattern(), Reason :: term().

Deletes all objects that match Pattern from table Name. For a description of patterns, see ets:match/2.

If the keypos'th element of Pattern is bound, only the objects with the correct key are matched.

match_object(Continuation)

-spec match_object(Continuation) -> {Objects, Continuation2} | '$end_of_table' | {error, Reason}
                      when
                          Continuation :: object_cont(),
                          Continuation2 :: object_cont(),
                          Objects :: [object()],
                          Reason :: term().

Returns a non-empty list of some objects stored in a table that match a given pattern in some unspecified order. The table, the pattern, and the number of objects that are matched are all defined by Continuation, which has been returned by a previous call to match_object/1 or match_object/3.

When all table objects are matched, '$end_of_table' is returned.

match_object(Name, Pattern)

-spec match_object(Name, Pattern) -> Objects | {error, Reason}
                      when
                          Name :: tab_name(),
                          Pattern :: pattern(),
                          Objects :: [object()],
                          Reason :: term().

Returns a list of all objects of table Name that match Pattern in some unspecified order. For a description of patterns, see ets:match/2.

If the keypos'th element of Pattern is unbound, all table objects are matched. If the keypos'th element of Pattern is bound, only the objects with the correct key are matched.

Using the match_object functions for traversing all table objects is more efficient than calling first/1 and next/2 or slot/2.

match_object(Name, Pattern, N)

-spec match_object(Name, Pattern, N) -> {Objects, Continuation} | '$end_of_table' | {error, Reason}
                      when
                          Name :: tab_name(),
                          Pattern :: pattern(),
                          N :: default | non_neg_integer(),
                          Continuation :: object_cont(),
                          Objects :: [object()],
                          Reason :: term().

Matches some or all objects stored in table Name and returns a non-empty list of the objects that match Pattern in some unspecified order. For a description of patterns, see ets:match/2.

A list of objects and a continuation is returned, unless the table is empty, in which case '$end_of_table' is returned. The continuation is to be used when matching further objects by calling match_object/1.

If the keypos'th element of Pattern is bound, all table objects are matched. If the keypos'th element is unbound, all table objects are matched, N objects at a time, until at least one object matches or the end of the table is reached. The default, indicated by giving N the value default, is to let the number of objects vary depending on the sizes of the objects. All matching objects with the same key are always returned in the same reply, which implies that more than N objects can sometimes be returned.

The table is always to be protected using safe_fixtable/2 before calling match_object/3, otherwise errors can occur when calling match_object/1.

member(Name, Key)

-spec member(Name, Key) -> boolean() | {error, Reason}
                when Name :: tab_name(), Key :: term(), Reason :: term().

Works like lookup/2, but does not return the objects. Returns true if one or more table elements has key Key, otherwise false.

next(Name, Key1)

-spec next(Name, Key1) -> Key2 | '$end_of_table' when Name :: tab_name(), Key1 :: term(), Key2 :: term().

Returns either the key following Key1 in table Name according to the internal order of the table, or '$end_of_table' if there is no next key.

If an error occurs, the process is exited with an error tuple {error, Reason}.

To find the first key in the table, use first/1.

open_file(Filename)

-spec open_file(Filename) -> {ok, Reference} | {error, Reason}
                   when Filename :: file:name(), Reference :: reference(), Reason :: term().

Opens an existing table. If the table is not properly closed, it is repaired. The returned reference is to be used as the table name. This function is most useful for debugging purposes.

open_file(Name, Args)

-spec open_file(Name, Args) -> {ok, Name} | {error, Reason}
                   when
                       Name :: tab_name(),
                       Args :: [OpenArg],
                       OpenArg ::
                           {access, access()} |
                           {auto_save, auto_save()} |
                           {estimated_no_objects, non_neg_integer()} |
                           {file, file:name()} |
                           {max_no_slots, no_slots()} |
                           {min_no_slots, no_slots()} |
                           {keypos, keypos()} |
                           {ram_file, boolean()} |
                           {repair, boolean() | force} |
                           {type, type()},
                       Reason :: term().

Opens a table. An empty Dets table is created if no file exists.

The atom Name is the table name. The table name must be provided in all subsequent operations on the table. The name can be used by other processes as well, and many processes can share one table.

If two processes open the same table by giving the same name and arguments, the table has two users. If one user closes the table, it remains open until the second user closes it.

Argument Args is a list of {Key, Val} tuples, where the following values are allowed:

{access, access/0 } - Existing tables can be opened in read-only mode. A table that is opened in read-only mode is not subjected to the automatic file reparation algorithm if it is later opened after a crash. Defaults to read_write.

{auto_save, auto_save/0 } - The autosave interval. If the interval is an integer Time, the table is flushed to disk whenever it is not accessed for Time milliseconds. A table that has been flushed requires no reparation when reopened after an uncontrolled emulator halt. If the interval is the atom infinity, autosave is disabled. Defaults to 180000 (3 minutes).

{estimated_no_objects, no_slots/0 } - Equivalent to option min_no_slots.

{file, file:name/0 } - The name of the file to be opened. Defaults to the table name.

{max_no_slots, no_slots/0 } - The maximum number of slots to be used. Defaults to 32 M, which is the maximal value. Notice that a higher value can increase the table fragmentation, and a smaller value can decrease the fragmentation, at the expense of execution time.

{min_no_slots, no_slots/0 } - Application performance can be enhanced with this flag by specifying, when the table is created, the estimated number of different keys to be stored in the table. Defaults to 256, which is the minimum value.

{keypos, keypos/0 } - The position of the element of each object to be used as key. Defaults to 1. The ability to explicitly state the key position is most convenient when we want to store Erlang records in which the first position of the record is the name of the record type.

{ram_file, boolean()} - Whether the table is to be kept in RAM. Keeping the table in RAM can sound like an anomaly, but can enhance the performance of applications that open a table, insert a set of objects, and then close the table. When the table is closed, its contents are written to the disk file. Defaults to false.

{repair, Value} - Value can be either a boolean/0 or the atom force. The flag specifies if the Dets server is to invoke the automatic file reparation algorithm. Defaults to true. If false is specified, no attempt is made to repair the file, and {error, {needs_repair, FileName}} is returned if the table must be repaired.

Value force means that a reparation is made even if the table is properly closed. This is a seldom needed option.

Option repair is ignored if the table is already open.

{type, type/0 } - The table type. Defaults to set.

pid2name(Pid)

-spec pid2name(Pid) -> {ok, Name} | undefined when Pid :: pid(), Name :: tab_name().

Returns the table name given the pid of a process that handles requests to a table, or undefined if there is no such table.

This function is meant to be used for debugging only.

repair_continuation(Continuation, MatchSpec)

-spec repair_continuation(Continuation, MatchSpec) -> Continuation2
                             when
                                 Continuation :: select_cont(),
                                 Continuation2 :: select_cont(),
                                 MatchSpec :: match_spec().

This function can be used to restore an opaque continuation returned by select/3 or select/1 if the continuation has passed through external term format (been sent between nodes or stored on disk).

The reason for this function is that continuation terms contain compiled match specifications and therefore are invalidated if converted to external term format. Given that the original match specification is kept intact, the continuation can be restored, meaning it can once again be used in subsequent select/1 calls even though it has been stored on disk or on another node.

For more information and examples, see the ets module.

Note

This function is rarely needed in application code. It is used by application Mnesia to provide distributed select/3 and select/1 sequences. A normal application would either use Mnesia or keep the continuation from being converted to external format.

The reason for not having an external representation of compiled match specifications is performance. It can be subject to change in future releases, while this interface remains for backward compatibility.

safe_fixtable(Name, Fix)

-spec safe_fixtable(Name, Fix) -> ok when Name :: tab_name(), Fix :: boolean().

If Fix is true, table Name is fixed (once more) by the calling process, otherwise the table is released. The table is also released when a fixing process terminates.

If many processes fix a table, the table remains fixed until all processes have released it or terminated. A reference counter is kept on a per process basis, and N consecutive fixes require N releases to release the table.

It is not guaranteed that calls to first/1, next/2, or select and match functions work as expected even if the table is fixed; the limited support for concurrency provided by the ets module is not yet provided by Dets. Fixing a table currently only disables resizing of the hash list of the table.

If objects have been added while the table was fixed, the hash list starts to grow when the table is released, which significantly slows down access to the table for a period of time.

select(Continuation)

-spec select(Continuation) -> {Selection, Continuation2} | '$end_of_table' | {error, Reason}
                when
                    Continuation :: select_cont(),
                    Continuation2 :: select_cont(),
                    Selection :: [term()],
                    Reason :: term().

Applies a match specification to some objects stored in a table and returns a non-empty list of the results. The table, the match specification, and the number of objects that are matched are all defined by Continuation, which is returned by a previous call to select/1 or select/3.

When all objects of the table have been matched, '$end_of_table' is returned.

select(Name, MatchSpec)

-spec select(Name, MatchSpec) -> Selection | {error, Reason}
                when
                    Name :: tab_name(),
                    MatchSpec :: match_spec(),
                    Selection :: [term()],
                    Reason :: term().

Returns the results of applying match specification MatchSpec to all or some objects stored in table Name. The order of the objects is not specified. For a description of match specifications, see the ERTS User's Guide.

If the keypos'th element of MatchSpec is unbound, the match specification is applied to all objects of the table. If the keypos'th element is bound, the match specification is applied to the objects with the correct key(s) only.

Using the select functions for traversing all objects of a table is more efficient than calling first/1 and next/2 or slot/2.

select(Name, MatchSpec, N)

-spec select(Name, MatchSpec, N) -> {Selection, Continuation} | '$end_of_table' | {error, Reason}
                when
                    Name :: tab_name(),
                    MatchSpec :: match_spec(),
                    N :: default | non_neg_integer(),
                    Continuation :: select_cont(),
                    Selection :: [term()],
                    Reason :: term().

Returns the results of applying match specification MatchSpec to some or all objects stored in table Name. The order of the objects is not specified. For a description of match specifications, see the ERTS User's Guide.

A tuple of the results of applying the match specification and a continuation is returned, unless the table is empty, in which case '$end_of_table' is returned. The continuation is to be used when matching more objects by calling select/1.

If the keypos'th element of MatchSpec is bound, the match specification is applied to all objects of the table with the correct key(s). If the keypos'th element of MatchSpec is unbound, the match specification is applied to all objects of the table, N objects at a time, until at least one object matches or the end of the table is reached. The default, indicated by giving N the value default, is to let the number of objects vary depending on the sizes of the objects. All objects with the same key are always handled at the same time, which implies that the match specification can be applied to more than N objects.

The table is always to be protected using safe_fixtable/2 before calling select/3, otherwise errors can occur when calling select/1.

select_delete(Name, MatchSpec)

-spec select_delete(Name, MatchSpec) -> N | {error, Reason}
                       when
                           Name :: tab_name(),
                           MatchSpec :: match_spec(),
                           N :: non_neg_integer(),
                           Reason :: term().

Deletes each object from table Name such that applying match specification MatchSpec to the object returns value true. For a description of match specifications, see the ERTS User's Guide. Returns the number of deleted objects.

If the keypos'th element of MatchSpec is bound, the match specification is applied to the objects with the correct key(s) only.

slot(Name, I)

-spec slot(Name, I) -> '$end_of_table' | Objects | {error, Reason}
              when Name :: tab_name(), I :: non_neg_integer(), Objects :: [object()], Reason :: term().

The objects of a table are distributed among slots, starting with slot 0 and ending with slot n. Returns the list of objects associated with slot I. If I > n, '$end_of_table' is returned.

sync(Name)

-spec sync(Name) -> ok | {error, Reason} when Name :: tab_name(), Reason :: term().

Ensures that all updates made to table Name are written to disk. This also applies to tables that have been opened with flag ram_file set to true. In this case, the contents of the RAM file are flushed to disk.

Notice that the space management data structures kept in RAM, the buddy system, is also written to the disk. This can take some time if the table is fragmented.

table(Name)

-spec table(Name) -> QueryHandle when Name :: tab_name(), QueryHandle :: qlc:query_handle().

Equivalent to table(Name, []).

table(Name, Options)

-spec table(Name, Options) -> QueryHandle
               when
                   Name :: tab_name(),
                   Options :: Option | [Option],
                   Option :: {n_objects, Limit} | {traverse, TraverseMethod},
                   Limit :: default | pos_integer(),
                   TraverseMethod :: first_next | select | {select, match_spec()},
                   QueryHandle :: qlc:query_handle().

Returns a Query List Comprehension (QLC) query handle. The qlc module provides a query language aimed mainly for Mnesia, but ETS tables, Dets tables, and lists are also recognized by qlc as sources of data. Calling dets:table/1,2 is the means to make Dets table Name usable to qlc.

When there are only simple restrictions on the key position, qlc uses dets:lookup/2 to look up the keys. When that is not possible, the whole table is traversed. Option traverse determines how this is done:

first_next - The table is traversed one key at a time by calling dets:first/1 and dets:next/2.

select - The table is traversed by calling dets:select/3 and dets:select/1. Option n_objects determines the number of objects returned (the third argument of select/3). The match specification (the second argument of select/3) is assembled by qlc:

Simple filters are translated into equivalent match specifications.

More complicated filters must be applied to all objects returned by select/3 given a match specification that matches all objects.

{select, match_spec/0 } - As for select, the table is traversed by calling dets:select/3 and dets:select/1. The difference is that the match specification is specified explicitly. This is how to state match specifications that cannot easily be expressed within the syntax provided by qlc.

The following example uses an explicit match specification to traverse the table:

1> dets:open_file(t, []),
ok = dets:insert(t, [{1,a},{2,b},{3,c},{4,d}]),
MS = ets:fun2ms(fun({X,Y}) when (X > 1) or (X < 5) -> {Y} end),
QH1 = dets:table(t, [{traverse, {select, MS}}]).

An example with implicit match specification:

2> QH2 = qlc:q([{Y} || {X,Y} <- dets:table(t), (X > 1) or (X < 5)]).

The latter example is equivalent to the former, which can be verified using function qlc:info/1:

3> qlc:info(QH1) =:= qlc:info(QH2).
true

qlc:info/1 returns information about a query handle. In this case identical information is returned for the two query handles.

to_ets(Name, EtsTab)

-spec to_ets(Name, EtsTab) -> EtsTab | {error, Reason}
                when Name :: tab_name(), EtsTab :: ets:table(), Reason :: term().

Inserts the objects of the Dets table Name into the ETS table EtsTab. The order in which the objects are inserted is not specified. The existing objects of the ETS table are kept unless overwritten.

traverse(Name, Fun)

-spec traverse(Name, Fun) -> Return | {error, Reason}
                  when
                      Name :: tab_name(),
                      Fun :: fun((Object) -> FunReturn),
                      Object :: object(),
                      FunReturn :: continue | {continue, Val} | {done, Value} | OtherValue,
                      Return :: [term()] | OtherValue,
                      Val :: term(),
                      Value :: term(),
                      OtherValue :: term(),
                      Reason :: term().

Applies Fun to each object stored in table Name in some unspecified order. Different actions are taken depending on the return value of Fun. The following Fun return values are allowed:

continue - Continue to perform the traversal. For example, the following function can be used to print the contents of a table:

fun(X) -> io:format("~p~n", [X]), continue end.

{continue, Val} - Continue the traversal and accumulate Val. The following function is supplied to collect all objects of a table in a list:

fun(X) -> {continue, X} end.

{done, Value} - Terminate the traversal and return [Value | Acc].

Any other value OtherValue returned by Fun terminates the traversal and is returned immediately.

update_counter(Name, Key, Increment)

-spec update_counter(Name, Key, Increment) -> Result
                        when
                            Name :: tab_name(),
                            Key :: term(),
                            Increment :: {Pos, Incr} | Incr,
                            Pos :: integer(),
                            Incr :: integer(),
                            Result :: integer().

Updates the object with key Key stored in table Name of type set by adding Incr to the element at the Pos:th position. The new counter value is returned. If no position is specified, the element directly following the key is updated.

This functions provides a way of updating a counter, without having to look up an object, update the object by incrementing an element, and insert the resulting object into the table again.

dict

A Key-value dictionary.

The representation of a dictionary is not defined.

This module provides the same interface as the orddict module. One difference is that while this module considers two keys as different if they do not match (=:=), orddict considers two keys as different if and only if they do not compare equal (==).

Notes

Functions append and append_list are included so that keyed values can be stored in a list accumulator, for example:

> D0 = dict:new(),
  D1 = dict:store(files, [], D0),
  D2 = dict:append(files, f1, D1),
  D3 = dict:append(files, f2, D2),
  D4 = dict:append(files, f3, D3),
  dict:fetch(files, D4).
[f1,f2,f3]

This saves the trouble of first fetching a keyed value, appending a new value to the list of stored values, and storing the result.

Function fetch is to be used if the key is known to be in the dictionary, otherwise function find.

See Also

gb_trees, orddict

Summary

Types

dict()

dict(Key, Value)

Dictionary as returned by new/0.

Functions

append(Key, Value, Dict1)

Appends a new Value to the current list of values associated with Key.

append_list(Key, ValList, Dict1)

Appends a list of values ValList to the current list of values associated with Key. An exception is generated if the initial value associated with Key is not a list of values.

erase(Key, Dict1)

Erases all items with a given key from a dictionary.

fetch(Key, Dict)

Returns the value associated with Key in dictionary Dict. This function assumes that Key is present in dictionary Dict, and an exception is generated if Key is not in the dictionary.

fetch_keys(Dict)

Returns a list of all keys in dictionary Dict.

filter(Pred, Dict1)

Dict2 is a dictionary of all keys and values in Dict1 for which Pred(Key, Value) is true.

find(Key, Dict)

Searches for a key in dictionary Dict. Returns {ok, Value}, where Value is the value associated with Key, or error if the key is not present in the dictionary.

fold(Fun, Acc0, Dict)

Calls Fun on successive keys and values of dictionary Dict together with an extra argument Acc (short for accumulator). Fun must return a new accumulator that is passed to the next call. Acc0 is returned if the dictionary is empty. The evaluation order is undefined.

from_list(List)

Converts the Key-Value list List to dictionary Dict.

is_empty(Dict)

Returns true if dictionary Dict has no elements, otherwise false.

is_key(Key, Dict)

Tests if Key is contained in dictionary Dict.

map(Fun, Dict1)

Calls Fun on successive keys and values of dictionary Dict1 to return a new value for each key. The evaluation order is undefined.

merge(Fun, Dict1, Dict2)

Merges two dictionaries, Dict1 and Dict2, to create a new dictionary. All the Key-Value pairs from both dictionaries are included in the new dictionary. If a key occurs in both dictionaries, Fun is called with the key and both values to return a new value. merge can be defined as follows, but is faster

new()

Creates a new dictionary.

size(Dict)

Returns the number of elements in dictionary Dict.

store(Key, Value, Dict1)

Stores a Key-Value pair in dictionary Dict2. If Key already exists in Dict1, the associated value is replaced by Value.

take(Key, Dict)

This function returns value from dictionary and a new dictionary without this value. Returns error if the key is not present in the dictionary.

to_list(Dict)

Converts dictionary Dict to a list representation.

update(Key, Fun, Dict1)

Updates a value in a dictionary by calling Fun on the value to get a new value. An exception is generated if Key is not present in the dictionary.

update(Key, Fun, Initial, Dict1)

Updates a value in a dictionary by calling Fun on the value to get a new value. If Key is not present in the dictionary, Initial is stored as the first value. For example, append/3 can be defined as

update_counter(Key, Increment, Dict1)

Adds Increment to the value associated with Key and stores this value. If Key is not present in the dictionary, Increment is stored as the first value.

Types

dict()

-type dict() :: dict(_, _).

dict(Key, Value)

-opaque dict(Key, Value)

Dictionary as returned by new/0.

Functions

append(Key, Value, Dict1)

-spec append(Key, Value, Dict1) -> Dict2 when Dict1 :: dict(Key, Value), Dict2 :: dict(Key, Value).

Appends a new Value to the current list of values associated with Key.

See also section Notes.

append_list(Key, ValList, Dict1)

-spec append_list(Key, ValList, Dict1) -> Dict2
                     when Dict1 :: dict(Key, Value), Dict2 :: dict(Key, Value), ValList :: [Value].

Appends a list of values ValList to the current list of values associated with Key. An exception is generated if the initial value associated with Key is not a list of values.

See also section Notes.

erase(Key, Dict1)

-spec erase(Key, Dict1) -> Dict2 when Dict1 :: dict(Key, Value), Dict2 :: dict(Key, Value).

Erases all items with a given key from a dictionary.

fetch(Key, Dict)

-spec fetch(Key, Dict) -> Value when Dict :: dict(Key, Value).

Returns the value associated with Key in dictionary Dict. This function assumes that Key is present in dictionary Dict, and an exception is generated if Key is not in the dictionary.

See also section Notes.

fetch_keys(Dict)

-spec fetch_keys(Dict) -> Keys when Dict :: dict(Key, Value :: term()), Keys :: [Key].

Returns a list of all keys in dictionary Dict.

filter(Pred, Dict1)

-spec filter(Pred, Dict1) -> Dict2
                when
                    Pred :: fun((Key, Value) -> boolean()),
                    Dict1 :: dict(Key, Value),
                    Dict2 :: dict(Key, Value).

Dict2 is a dictionary of all keys and values in Dict1 for which Pred(Key, Value) is true.

find(Key, Dict)

-spec find(Key, Dict) -> {ok, Value} | error when Dict :: dict(Key, Value).

Searches for a key in dictionary Dict. Returns {ok, Value}, where Value is the value associated with Key, or error if the key is not present in the dictionary.

See also section Notes.

fold(Fun, Acc0, Dict)

-spec fold(Fun, Acc0, Dict) -> Acc1
              when
                  Fun :: fun((Key, Value, AccIn) -> AccOut),
                  Dict :: dict(Key, Value),
                  Acc0 :: Acc,
                  Acc1 :: Acc,
                  AccIn :: Acc,
                  AccOut :: Acc.

Calls Fun on successive keys and values of dictionary Dict together with an extra argument Acc (short for accumulator). Fun must return a new accumulator that is passed to the next call. Acc0 is returned if the dictionary is empty. The evaluation order is undefined.

from_list(List)

-spec from_list(List) -> Dict when Dict :: dict(Key, Value), List :: [{Key, Value}].

Converts the Key-Value list List to dictionary Dict.

is_empty(Dict)

(since OTP 17.0)

-spec is_empty(Dict) -> boolean() when Dict :: dict().

Returns true if dictionary Dict has no elements, otherwise false.

is_key(Key, Dict)

-spec is_key(Key, Dict) -> boolean() when Dict :: dict(Key, Value :: term()).

Tests if Key is contained in dictionary Dict.

map(Fun, Dict1)

-spec map(Fun, Dict1) -> Dict2
             when
                 Fun :: fun((Key, Value1) -> Value2),
                 Dict1 :: dict(Key, Value1),
                 Dict2 :: dict(Key, Value2).

Calls Fun on successive keys and values of dictionary Dict1 to return a new value for each key. The evaluation order is undefined.

merge(Fun, Dict1, Dict2)

-spec merge(Fun, Dict1, Dict2) -> Dict3
               when
                   Fun :: fun((Key, Value1, Value2) -> Value),
                   Dict1 :: dict(Key, Value1),
                   Dict2 :: dict(Key, Value2),
                   Dict3 :: dict(Key, Value).

Merges two dictionaries, Dict1 and Dict2, to create a new dictionary. All the Key-Value pairs from both dictionaries are included in the new dictionary. If a key occurs in both dictionaries, Fun is called with the key and both values to return a new value. merge can be defined as follows, but is faster:

merge(Fun, D1, D2) ->
    fold(fun (K, V1, D) ->
                 update(K, fun (V2) -> Fun(K, V1, V2) end, V1, D)
         end, D2, D1).

new()

-spec new() -> dict().

Creates a new dictionary.

size(Dict)

-spec size(Dict) -> non_neg_integer() when Dict :: dict().

Returns the number of elements in dictionary Dict.

store(Key, Value, Dict1)

-spec store(Key, Value, Dict1) -> Dict2 when Dict1 :: dict(Key, Value), Dict2 :: dict(Key, Value).

Stores a Key-Value pair in dictionary Dict2. If Key already exists in Dict1, the associated value is replaced by Value.

take(Key, Dict)

(since OTP 20.0)

-spec take(Key, Dict) -> {Value, Dict1} | error
              when Dict :: dict(Key, Value), Dict1 :: dict(Key, Value), Key :: term(), Value :: term().

This function returns value from dictionary and a new dictionary without this value. Returns error if the key is not present in the dictionary.

to_list(Dict)

-spec to_list(Dict) -> List when Dict :: dict(Key, Value), List :: [{Key, Value}].

Converts dictionary Dict to a list representation.

update(Key, Fun, Dict1)

-spec update(Key, Fun, Dict1) -> Dict2
                when
                    Dict1 :: dict(Key, Value),
                    Dict2 :: dict(Key, Value),
                    Fun :: fun((Value1 :: Value) -> Value2 :: Value).

Updates a value in a dictionary by calling Fun on the value to get a new value. An exception is generated if Key is not present in the dictionary.

update(Key, Fun, Initial, Dict1)

-spec update(Key, Fun, Initial, Dict1) -> Dict2
                when
                    Dict1 :: dict(Key, Value),
                    Dict2 :: dict(Key, Value),
                    Fun :: fun((Value1 :: Value) -> Value2 :: Value),
                    Initial :: Value.

Updates a value in a dictionary by calling Fun on the value to get a new value. If Key is not present in the dictionary, Initial is stored as the first value. For example, append/3 can be defined as:

append(Key, Val, D) ->
    update(Key, fun (Old) -> Old ++ [Val] end, [Val], D).

update_counter(Key, Increment, Dict1)

-spec update_counter(Key, Increment, Dict1) -> Dict2
                        when Dict1 :: dict(Key, Value), Dict2 :: dict(Key, Value), Increment :: number().

Adds Increment to the value associated with Key and stores this value. If Key is not present in the dictionary, Increment is stored as the first value.

This can be defined as follows, but is faster:

update_counter(Key, Incr, D) ->
    update(Key, fun (Old) -> Old + Incr end, Incr, D).

digraph

This module provides a version of labeled directed graphs ("digraphs").

The digraphs managed by this module are stored in ETS tables. That implies the following:

Only the process that created the digraph is allowed to update it.

Digraphs will not be garbage collected. The ETS tables used for a digraph will only be deleted when delete/1 is called or the process that created the digraph terminates.

A digraph is a mutable data structure.

What makes the graphs provided here non-proper directed graphs is that multiple edges between vertices are allowed. However, the customary definition of directed graphs is used here.

A directed graph (or just "digraph") is a pair (V, E) of a finite set V of vertices and a finite set E of directed edges (or just "edges"). The set of edges E is a subset of V × V (the Cartesian product of V with itself).

In this module, V is allowed to be empty. The so obtained unique digraph is called the empty digraph. Both vertices and edges are represented by unique Erlang terms.

Digraphs can be annotated with more information. Such information can be attached to the vertices and to the edges of the digraph. An annotated digraph is called a labeled digraph, and the information attached to a vertex or an edge is called a label. Labels are Erlang terms.

An edge e = (v, w) is said to emanate from vertex v and to be incident on vertex w.

The out-degree of a vertex is the number of edges emanating from that vertex.

The in-degree of a vertex is the number of edges incident on that vertex.

If an edge is emanating from v and incident on w, then w is said to be an out-neighbor of v, and v is said to be an in-neighbor of w.

A path P from v[1] to v[k] in a digraph (V, E) is a non-empty sequence v[1], v[2], ..., v[k] of vertices in V such that there is an edge (v[i],v[i+1]) in E for 1 <= i < k.

The length of path P is k-1.

Path P is simple if all vertices are distinct, except that the first and the last vertices can be the same.

Path P is a cycle if the length of P is not zero and v[1] = v[k].

A loop is a cycle of length one.

A simple cycle is a path that is both a cycle and simple.

An acyclic digraph is a digraph without cycles.

See Also

digraph_utils, ets

Summary

Types

add_edge_err_rsn()

The error reason for when an edge could not be added to a graph.

d_cyclicity()

d_protection()

d_type()

edge()

Serves as the identifier or "name" of an edge. This is distinct from an edge "label" which attaches ancillary information to the edge rather than identifying the edge itself.

graph()

A digraph as returned by new/0,1.

label()

vertex()

Functions

add_edge(G, V1, V2)

Equivalent to add_edge(G, V1, V2, []).

add_edge(G, V1, V2, Label)

Equivalent to add_edge(G, E, V1, V2, Label), where E is a created edge.

add_edge(G, E, V1, V2, Label)

Creates (or modifies) an edge with the identifier E of digraph G, using Label as the (new) label of the edge. The edge is emanating from V1 and incident on V2. Returns E.

add_vertex(G)

Creates a vertex using the empty list as label, and returns the created vertex.

add_vertex(G, V)

Equivalent to add_vertex(G, V, []).

add_vertex(G, V, Label)

Creates (or modifies) vertex V of digraph G, using Label as the (new) label of the vertex. Returns the new vertex V.

del_edge(G, E)

Deletes edge E from digraph G.

del_edges(G, Edges)

Deletes the edges in list Edges from digraph G.

del_path(G, V1, V2)

Deletes edges from digraph G until there are no paths from vertex V1 to vertex V2.

del_vertex(G, V)

Deletes vertex V from digraph G. Any edges emanating from V or incident on V are also deleted.

del_vertices(G, Vertices)

Deletes the vertices in list Vertices from digraph G.

delete(G)

Deletes digraph G. This call is important as digraphs are implemented with ETS. There is no garbage collection of ETS tables. However, the digraph is deleted if the process that created the digraph terminates.

edge(G, E)

Returns {E, V1, V2, Label}, where Label is the label of edge E emanating from V1 and incident on V2 of digraph G. If no edge E of digraph G exists, false is returned.

edges(G)

Returns a list of all edges of digraph G, in some unspecified order.

edges(G, V)

Returns a list of all edges emanating from or incident on V of digraph G, in some unspecified order.

get_cycle(G, V)

If a simple cycle of length two or more exists through vertex V, the cycle is returned as a list [V, ..., V] of vertices. If a loop through V exists, the loop is returned as a list [V]. If no cycles through V exist, false is returned.

get_path(G, V1, V2)

Tries to find a simple path from vertex V1 to vertex V2 of digraph G. Returns the path as a list [V1, ..., V2] of vertices, or false if no simple path from V1 to V2 of length one or more exists.

get_short_cycle(G, V)

Tries to find an as short as possible simple cycle through vertex V of digraph G. Returns the cycle as a list [V, ..., V] of vertices, or false if no simple cycle through V exists. Notice that a loop through V is returned as list [V, V].

get_short_path(G, V1, V2)

Tries to find an as short as possible simple path from vertex V1 to vertex V2 of digraph G. Returns the path as a list [V1, ..., V2] of vertices, or false if no simple path from V1 to V2 of length one or more exists.

in_degree(G, V)

Returns the in-degree of vertex V of digraph G.

in_edges(G, V)

Returns a list of all edges incident on V of digraph G, in some unspecified order.

in_neighbours(G, V)

Returns a list of all in-neighbors of V of digraph G, in some unspecified order.

info(G)

Returns a list of {Tag, Value} pairs describing digraph G. The following pairs are returned

new()

Equivalent to new([]).

new(Type)

Returns an empty digraph with properties according to the options in Type

no_edges(G)

Returns the number of edges of digraph G.

no_vertices(G)

Returns the number of vertices of digraph G.

out_degree(G, V)

Returns the out-degree of vertex V of digraph G.

out_edges(G, V)

Returns a list of all edges emanating from V of digraph G, in some unspecified order.

out_neighbours(G, V)

Returns a list of all out-neighbors of V of digraph G, in some unspecified order.

vertex(G, V)

Returns {V, Label}, where Label is the label of the vertex V of digraph G, or false if no vertex V of digraph G exists.

vertices(G)

Returns a list of all vertices of digraph G, in some unspecified order.

Types

add_edge_err_rsn()

(not exported)

-type add_edge_err_rsn() :: {bad_edge, Path :: [vertex()]} | {bad_vertex, V :: vertex()}.

The error reason for when an edge could not be added to a graph.

If the edge would create a cycle in an acyclic digraph, {error, {bad_edge, Path}} is returned. If G already has an edge with value E connecting a different pair of vertices, {error, {bad_edge, [V1, V2]}} is returned. If either of V1 or V2 is not a vertex of digraph G, {error, {bad_vertex,V}} is returned, V = V1 or V = V2.

d_cyclicity()

(not exported)

-type d_cyclicity() :: acyclic | cyclic.

d_protection()

(not exported)

-type d_protection() :: private | protected.

d_type()

-type d_type() :: d_cyclicity() | d_protection().

edge()

-type edge() :: term().

Serves as the identifier or "name" of an edge. This is distinct from an edge "label" which attaches ancillary information to the edge rather than identifying the edge itself.

graph()

-opaque graph()

A digraph as returned by new/0,1.

label()

-type label() :: term().

vertex()

-type vertex() :: term().

Functions

add_edge(G, V1, V2)

-spec add_edge(G, V1, V2) -> edge() | {error, add_edge_err_rsn()}
                  when G :: graph(), V1 :: vertex(), V2 :: vertex().

Equivalent to add_edge(G, V1, V2, []).

add_edge(G, V1, V2, Label)

-spec add_edge(G, V1, V2, Label) -> edge() | {error, add_edge_err_rsn()}
                  when G :: graph(), V1 :: vertex(), V2 :: vertex(), Label :: label().

Equivalent to add_edge(G, E, V1, V2, Label), where E is a created edge.

The created edge is represented by term ['$e' | N], where N is an integer >= 0.

See add_edge_err_rsn/0 for details on possible errors.

add_edge(G, E, V1, V2, Label)

-spec add_edge(G, E, V1, V2, Label) -> edge() | {error, add_edge_err_rsn()}
                  when G :: graph(), E :: edge(), V1 :: vertex(), V2 :: vertex(), Label :: label().

Creates (or modifies) an edge with the identifier E of digraph G, using Label as the (new) label of the edge. The edge is emanating from V1 and incident on V2. Returns E.

See add_edge_err_rsn/0 for details on possible errors.

add_vertex(G)

-spec add_vertex(G) -> vertex() when G :: graph().

Creates a vertex using the empty list as label, and returns the created vertex.

The created vertex is represented by term ['$v' | N], where N is an integer >= 0.

add_vertex(G, V)

-spec add_vertex(G, V) -> vertex() when G :: graph(), V :: vertex().

Equivalent to add_vertex(G, V, []).

add_vertex(G, V, Label)

-spec add_vertex(G, V, Label) -> vertex() when G :: graph(), V :: vertex(), Label :: label().

Creates (or modifies) vertex V of digraph G, using Label as the (new) label of the vertex. Returns the new vertex V.

del_edge(G, E)

-spec del_edge(G, E) -> true when G :: graph(), E :: edge().

Deletes edge E from digraph G.

del_edges(G, Edges)

-spec del_edges(G, Edges) -> true when G :: graph(), Edges :: [edge()].

Deletes the edges in list Edges from digraph G.

del_path(G, V1, V2)

-spec del_path(G, V1, V2) -> true when G :: graph(), V1 :: vertex(), V2 :: vertex().

Deletes edges from digraph G until there are no paths from vertex V1 to vertex V2.

A sketch of the procedure employed:

Find an arbitrary simple path v[1], v[2], ..., v[k] from V1 to V2 in G.

Remove all edges of G emanating from v[i] and incident to v[i+1] for 1 <= i < k (including multiple edges).

Repeat until there is no path between V1 and V2.

del_vertex(G, V)

-spec del_vertex(G, V) -> true when G :: graph(), V :: vertex().

Deletes vertex V from digraph G. Any edges emanating from V or incident on V are also deleted.

del_vertices(G, Vertices)

-spec del_vertices(G, Vertices) -> true when G :: graph(), Vertices :: [vertex()].

Deletes the vertices in list Vertices from digraph G.

delete(G)

-spec delete(G) -> true when G :: graph().

Deletes digraph G. This call is important as digraphs are implemented with ETS. There is no garbage collection of ETS tables. However, the digraph is deleted if the process that created the digraph terminates.

edge(G, E)

-spec edge(G, E) -> {E, V1, V2, Label} | false
              when G :: graph(), E :: edge(), V1 :: vertex(), V2 :: vertex(), Label :: label().

Returns {E, V1, V2, Label}, where Label is the label of edge E emanating from V1 and incident on V2 of digraph G. If no edge E of digraph G exists, false is returned.

edges(G)

-spec edges(G) -> Edges when G :: graph(), Edges :: [edge()].

Returns a list of all edges of digraph G, in some unspecified order.

edges(G, V)

-spec edges(G, V) -> Edges when G :: graph(), V :: vertex(), Edges :: [edge()].

Returns a list of all edges emanating from or incident on V of digraph G, in some unspecified order.

get_cycle(G, V)

-spec get_cycle(G, V) -> Vertices | false when G :: graph(), V :: vertex(), Vertices :: [vertex(), ...].

If a simple cycle of length two or more exists through vertex V, the cycle is returned as a list [V, ..., V] of vertices. If a loop through V exists, the loop is returned as a list [V]. If no cycles through V exist, false is returned.

get_path/3 is used for finding a simple cycle through V.

get_path(G, V1, V2)

-spec get_path(G, V1, V2) -> Vertices | false
                  when G :: graph(), V1 :: vertex(), V2 :: vertex(), Vertices :: [vertex(), ...].

Tries to find a simple path from vertex V1 to vertex V2 of digraph G. Returns the path as a list [V1, ..., V2] of vertices, or false if no simple path from V1 to V2 of length one or more exists.

Digraph G is traversed in a depth-first manner, and the first found path is returned.

get_short_cycle(G, V)

-spec get_short_cycle(G, V) -> Vertices | false
                         when G :: graph(), V :: vertex(), Vertices :: [vertex(), ...].

Tries to find an as short as possible simple cycle through vertex V of digraph G. Returns the cycle as a list [V, ..., V] of vertices, or false if no simple cycle through V exists. Notice that a loop through V is returned as list [V, V].

get_short_path/3 is used for finding a simple cycle through V.

get_short_path(G, V1, V2)

-spec get_short_path(G, V1, V2) -> Vertices | false
                        when G :: graph(), V1 :: vertex(), V2 :: vertex(), Vertices :: [vertex(), ...].

Tries to find an as short as possible simple path from vertex V1 to vertex V2 of digraph G. Returns the path as a list [V1, ..., V2] of vertices, or false if no simple path from V1 to V2 of length one or more exists.

Digraph G is traversed in a breadth-first manner, and the first found path is returned.

in_degree(G, V)

-spec in_degree(G, V) -> non_neg_integer() when G :: graph(), V :: vertex().

Returns the in-degree of vertex V of digraph G.

in_edges(G, V)

-spec in_edges(G, V) -> Edges when G :: graph(), V :: vertex(), Edges :: [edge()].

Returns a list of all edges incident on V of digraph G, in some unspecified order.

in_neighbours(G, V)

-spec in_neighbours(G, V) -> Vertex when G :: graph(), V :: vertex(), Vertex :: [vertex()].

Returns a list of all in-neighbors of V of digraph G, in some unspecified order.

info(G)

-spec info(G) -> InfoList
              when
                  G :: graph(),
                  InfoList ::
                      [{cyclicity, Cyclicity :: d_cyclicity()} |
                       {memory, NoWords :: non_neg_integer()} |
                       {protection, Protection :: d_protection()}].

Returns a list of {Tag, Value} pairs describing digraph G. The following pairs are returned:

{cyclicity, Cyclicity}, where Cyclicity is cyclic or acyclic, according to the options given to new.

{memory, NoWords}, where NoWords is the number of words allocated to the ETS tables.

{protection, Protection}, where Protection is protected or private, according to the options given to new.

new()

-spec new() -> graph().

Equivalent to new([]).

new(Type)

-spec new(Type) -> graph() when Type :: [d_type()].

Returns an empty digraph with properties according to the options in Type:

cyclic - Allows cycles in the digraph (default).

acyclic - The digraph is to be kept acyclic.

protected - Other processes can read the digraph (default).

private - The digraph can be read and modified by the creating process only.

If an unrecognized type option T is specified or Type is not a proper list, a badarg exception is raised.

no_edges(G)

-spec no_edges(G) -> non_neg_integer() when G :: graph().

Returns the number of edges of digraph G.

no_vertices(G)

-spec no_vertices(G) -> non_neg_integer() when G :: graph().

Returns the number of vertices of digraph G.

out_degree(G, V)

-spec out_degree(G, V) -> non_neg_integer() when G :: graph(), V :: vertex().

Returns the out-degree of vertex V of digraph G.

out_edges(G, V)

-spec out_edges(G, V) -> Edges when G :: graph(), V :: vertex(), Edges :: [edge()].

Returns a list of all edges emanating from V of digraph G, in some unspecified order.

out_neighbours(G, V)

-spec out_neighbours(G, V) -> Vertices when G :: graph(), V :: vertex(), Vertices :: [vertex()].

Returns a list of all out-neighbors of V of digraph G, in some unspecified order.

vertex(G, V)

-spec vertex(G, V) -> {V, Label} | false when G :: graph(), V :: vertex(), Label :: label().

Returns {V, Label}, where Label is the label of the vertex V of digraph G, or false if no vertex V of digraph G exists.

vertices(G)

-spec vertices(G) -> Vertices when G :: graph(), Vertices :: [vertex()].

Returns a list of all vertices of digraph G, in some unspecified order.

digraph_utils

This module provides algorithms based on depth-first traversal of directed graphs.

For basic functions on directed graphs, see the digraph module.

A directed graph (or just "digraph") is a pair (V, E) of a finite set V of vertices and a finite set E of directed edges (or just "edges"). The set of edges E is a subset of V × V (the Cartesian product of V with itself).

Digraphs can be annotated with more information. Such information can be attached to the vertices and to the edges of the digraph. An annotated digraph is called a labeled digraph, and the information attached to a vertex or an edge is called a label.

An edge e = (v, w) is said to emanate from vertex v and to be incident on vertex w.

If an edge is emanating from v and incident on w, then w is said to be an out-neighbor of v, and v is said to be an in-neighbor of w.

A path P from v[1] to v[k] in a digraph (V, E) is a non-empty sequence v[1], v[2], ..., v[k] of vertices in V such that there is an edge (v[i],v[i+1]) in E for 1 <= i < k.

The length of path P is k-1.

Path P is a cycle if the length of P is not zero and v[1] = v[k].

A loop is a cycle of length one.

An acyclic digraph is a digraph without cycles.

A depth-first traversal of a directed digraph can be viewed as a process that visits all vertices of the digraph. Initially, all vertices are marked as unvisited. The traversal starts with an arbitrarily chosen vertex, which is marked as visited, and follows an edge to an unmarked vertex, marking that vertex. The search then proceeds from that vertex in the same fashion, until there is no edge leading to an unvisited vertex. At that point the process backtracks, and the traversal continues as long as there are unexamined edges. If unvisited vertices remain when all edges from the first vertex have been examined, some so far unvisited vertex is chosen, and the process is repeated.

A partial ordering of a set S is a transitive, antisymmetric, and reflexive relation between the objects of S.

The problem of topological sorting is to find a total ordering of S that is a superset of the partial ordering. A digraph G = (V, E) is equivalent to a relation E on V (we neglect that the version of directed graphs provided by the digraph module allows multiple edges between vertices). If the digraph has no cycles of length two or more, the reflexive and transitive closure of E is a partial ordering.

A subgraph G' of G is a digraph whose vertices and edges form subsets of the vertices and edges of G.

G' is maximal with respect to a property P if all other subgraphs that include the vertices of G' do not have property P.

A strongly connected component is a maximal subgraph such that there is a path between each pair of vertices

A connected component is a maximal subgraph such that there is a path between each pair of vertices, considering all edges undirected.

An arborescence is an acyclic digraph with a vertex V, the root, such that there is a unique path from V to every other vertex of G.

A tree is an acyclic non-empty digraph such that there is a unique path between every pair of vertices, considering all edges undirected.

See Also

digraph

Summary

Functions

arborescence_root(Digraph)

Returns {yes, Root} if Root is the root of the arborescence Digraph, otherwise no.

components(Digraph)

Returns a list of connected components. Each component is represented by its vertices. The order of the vertices and the order of the components are arbitrary. Each vertex of digraph Digraph occurs in exactly one component.

condensation(Digraph)

Creates a digraph where the vertices are the strongly connected components of Digraph as returned by strong_components/1. If X and Y are two different strongly connected components, and vertices x and y exist in X and Y, respectively, such that there is an edge emanating from x and incident on y, then an edge emanating from X and incident on Y is created.

cyclic_strong_components(Digraph)

Returns a list of strongly connected components. Each strongly component is represented by its vertices. The order of the vertices and the order of the components are arbitrary. Only vertices that are included in some cycle in Digraph are returned, otherwise the returned list is equal to that returned by strong_components/1.

is_acyclic(Digraph)

Returns true if and only if digraph Digraph is acyclic.

is_arborescence(Digraph)

Returns true if and only if digraph Digraph is an arborescence.

is_tree(Digraph)

Returns true if and only if digraph Digraph is a tree.

loop_vertices(Digraph)

Returns a list of all vertices of Digraph that are included in some loop.

postorder(Digraph)

Returns all vertices of digraph Digraph. The order is given by a depth-first traversal of the digraph, collecting visited vertices in postorder. More precisely, the vertices visited while searching from an arbitrarily chosen vertex are collected in postorder, and all those collected vertices are placed before the subsequently visited vertices.

preorder(Digraph)

Returns all vertices of digraph Digraph. The order is given by a depth-first traversal of the digraph, collecting visited vertices in preorder.

reachable(Vertices, Digraph)

Returns an unsorted list of digraph vertices such that for each vertex in the list, there is a path in Digraph from some vertex of Vertices to the vertex. In particular, as paths can have length zero, the vertices of Vertices are included in the returned list.

reachable_neighbours(Vertices, Digraph)

Returns an unsorted list of digraph vertices such that for each vertex in the list, there is a path in Digraph of length one or more from some vertex of Vertices to the vertex. As a consequence, only those vertices of Vertices that are included in some cycle are returned.

reaching(Vertices, Digraph)

Returns an unsorted list of digraph vertices such that for each vertex in the list, there is a path from the vertex to some vertex of Vertices. In particular, as paths can have length zero, the vertices of Vertices are included in the returned list.

reaching_neighbours(Vertices, Digraph)

Returns an unsorted list of digraph vertices such that for each vertex in the list, there is a path of length one or more from the vertex to some vertex of Vertices. Therefore only those vertices of Vertices that are included in some cycle are returned.

strong_components(Digraph)

Returns a list of strongly connected components. Each strongly component is represented by its vertices. The order of the vertices and the order of the components are arbitrary. Each vertex of digraph Digraph occurs in exactly one strong component.

subgraph(Digraph, Vertices)

Equivalent to subgraph/3.

subgraph(Digraph, Vertices, Options)

Creates a maximal subgraph of Digraph having as vertices those vertices of Digraph that are mentioned in Vertices.

topsort(Digraph)

Returns a topological ordering of the vertices of digraph Digraph if such an ordering exists, otherwise false. For each vertex in the returned list, no out-neighbors occur earlier in the list.

Functions

arborescence_root(Digraph)

-spec arborescence_root(Digraph) -> no | {yes, Root}
                           when Digraph :: digraph:graph(), Root :: digraph:vertex().

Returns {yes, Root} if Root is the root of the arborescence Digraph, otherwise no.

components(Digraph)

-spec components(Digraph) -> [Component]
                    when Digraph :: digraph:graph(), Component :: [digraph:vertex()].

Returns a list of connected components. Each component is represented by its vertices. The order of the vertices and the order of the components are arbitrary. Each vertex of digraph Digraph occurs in exactly one component.

condensation(Digraph)

-spec condensation(Digraph) -> CondensedDigraph
                      when Digraph :: digraph:graph(), CondensedDigraph :: digraph:graph().

Creates a digraph where the vertices are the strongly connected components of Digraph as returned by strong_components/1. If X and Y are two different strongly connected components, and vertices x and y exist in X and Y, respectively, such that there is an edge emanating from x and incident on y, then an edge emanating from X and incident on Y is created.

The created digraph has the same type as Digraph. All vertices and edges have the default label [].

Each cycle is included in some strongly connected component, which implies that a topological ordering of the created digraph always exists.

cyclic_strong_components(Digraph)

-spec cyclic_strong_components(Digraph) -> [StrongComponent]
                                  when Digraph :: digraph:graph(), StrongComponent :: [digraph:vertex()].

Returns a list of strongly connected components. Each strongly component is represented by its vertices. The order of the vertices and the order of the components are arbitrary. Only vertices that are included in some cycle in Digraph are returned, otherwise the returned list is equal to that returned by strong_components/1.

is_acyclic(Digraph)

-spec is_acyclic(Digraph) -> boolean() when Digraph :: digraph:graph().

Returns true if and only if digraph Digraph is acyclic.

is_arborescence(Digraph)

-spec is_arborescence(Digraph) -> boolean() when Digraph :: digraph:graph().

Returns true if and only if digraph Digraph is an arborescence.

is_tree(Digraph)

-spec is_tree(Digraph) -> boolean() when Digraph :: digraph:graph().

Returns true if and only if digraph Digraph is a tree.

loop_vertices(Digraph)

-spec loop_vertices(Digraph) -> Vertices when Digraph :: digraph:graph(), Vertices :: [digraph:vertex()].

Returns a list of all vertices of Digraph that are included in some loop.

postorder(Digraph)

-spec postorder(Digraph) -> Vertices when Digraph :: digraph:graph(), Vertices :: [digraph:vertex()].

Returns all vertices of digraph Digraph. The order is given by a depth-first traversal of the digraph, collecting visited vertices in postorder. More precisely, the vertices visited while searching from an arbitrarily chosen vertex are collected in postorder, and all those collected vertices are placed before the subsequently visited vertices.

preorder(Digraph)

-spec preorder(Digraph) -> Vertices when Digraph :: digraph:graph(), Vertices :: [digraph:vertex()].

Returns all vertices of digraph Digraph. The order is given by a depth-first traversal of the digraph, collecting visited vertices in preorder.

reachable(Vertices, Digraph)

-spec reachable(Vertices, Digraph) -> Reachable
                   when
                       Digraph :: digraph:graph(),
                       Vertices :: [digraph:vertex()],
                       Reachable :: [digraph:vertex()].

Returns an unsorted list of digraph vertices such that for each vertex in the list, there is a path in Digraph from some vertex of Vertices to the vertex. In particular, as paths can have length zero, the vertices of Vertices are included in the returned list.

reachable_neighbours(Vertices, Digraph)

-spec reachable_neighbours(Vertices, Digraph) -> Reachable
                              when
                                  Digraph :: digraph:graph(),
                                  Vertices :: [digraph:vertex()],
                                  Reachable :: [digraph:vertex()].

Returns an unsorted list of digraph vertices such that for each vertex in the list, there is a path in Digraph of length one or more from some vertex of Vertices to the vertex. As a consequence, only those vertices of Vertices that are included in some cycle are returned.

reaching(Vertices, Digraph)

-spec reaching(Vertices, Digraph) -> Reaching
                  when
                      Digraph :: digraph:graph(),
                      Vertices :: [digraph:vertex()],
                      Reaching :: [digraph:vertex()].

Returns an unsorted list of digraph vertices such that for each vertex in the list, there is a path from the vertex to some vertex of Vertices. In particular, as paths can have length zero, the vertices of Vertices are included in the returned list.

reaching_neighbours(Vertices, Digraph)

-spec reaching_neighbours(Vertices, Digraph) -> Reaching
                             when
                                 Digraph :: digraph:graph(),
                                 Vertices :: [digraph:vertex()],
                                 Reaching :: [digraph:vertex()].

Returns an unsorted list of digraph vertices such that for each vertex in the list, there is a path of length one or more from the vertex to some vertex of Vertices. Therefore only those vertices of Vertices that are included in some cycle are returned.

strong_components(Digraph)

-spec strong_components(Digraph) -> [StrongComponent]
                           when Digraph :: digraph:graph(), StrongComponent :: [digraph:vertex()].

Returns a list of strongly connected components. Each strongly component is represented by its vertices. The order of the vertices and the order of the components are arbitrary. Each vertex of digraph Digraph occurs in exactly one strong component.

subgraph(Digraph, Vertices)

-spec subgraph(Digraph, Vertices) -> SubGraph
                  when
                      Digraph :: digraph:graph(),
                      Vertices :: [digraph:vertex()],
                      SubGraph :: digraph:graph().

Equivalent to subgraph/3.

subgraph(Digraph, Vertices, Options)

-spec subgraph(Digraph, Vertices, Options) -> SubGraph
                  when
                      Digraph :: digraph:graph(),
                      SubGraph :: digraph:graph(),
                      Vertices :: [digraph:vertex()],
                      Options :: [{type, SubgraphType} | {keep_labels, boolean()}],
                      SubgraphType :: inherit | [digraph:d_type()].

Creates a maximal subgraph of Digraph having as vertices those vertices of Digraph that are mentioned in Vertices.

If the value of option type is inherit, which is the default, the type of Digraph is used for the subgraph as well. Otherwise the option value of type is used as argument to digraph:new/1.

If the value of option keep_labels is true, which is the default, the labels of vertices and edges of Digraph are used for the subgraph as well. If the value is false, default label [] is used for the vertices and edges of the subgroup.

subgraph(Digraph, Vertices) is equivalent to subgraph(Digraph, Vertices, []).

If any of the arguments are invalid, a badarg exception is raised.

topsort(Digraph)

-spec topsort(Digraph) -> Vertices | false
                 when Digraph :: digraph:graph(), Vertices :: [digraph:vertex()].

Returns a topological ordering of the vertices of digraph Digraph if such an ordering exists, otherwise false. For each vertex in the returned list, no out-neighbors occur earlier in the list.

ets

Built-in term storage.

This module is an interface to the Erlang built-in term storage BIFs. These provide the ability to store very large quantities of data in an Erlang runtime system, and to have constant access time to the data. (In the case of ordered_set, see below, access time is proportional to the logarithm of the number of stored objects.)

Data is organized as a set of dynamic tables, which can store tuples. Each table is created by a process. When the process terminates, the table is automatically destroyed. Every table has access rights set at creation.

Tables are divided into four different types, set, ordered_set, bag, and duplicate_bag. A set or ordered_set table can only have one object associated with each key. A bag or duplicate_bag table can have many objects associated with each key.

Insert and lookup times in tables of type set are constant, regardless of the table size. For table types bag and duplicate_bag time is proportional to the number of objects with the same key. Even seemingly unrelated keys may inflict linear search to be skipped past while looking for the key of interest (due to hash collision).

Warning

For tables of type bag and duplicate_bag, avoid inserting an extensive amount of objects with the same key. It will hurt insert and lookup performance as well as real time characteristics of the runtime environment (hash bucket linear search do not yield).

The ordered_set table type uses a binary search tree. Insert and lookup times are proportional to the logarithm of the number of objects in the table.

Note

The number of tables stored at one Erlang node used to be limited. This is no longer the case (except by memory usage). The previous default limit was about 1400 tables and could be increased by setting the environment variable ERL_MAX_ETS_TABLES or the command line option +e before starting the Erlang runtime system. This hard limit has been removed, but it is currently useful to set the ERL_MAX_ETS_TABLES anyway. It should be set to an approximate of the maximum amount of tables used since an internal table for named tables is sized using this value. If large amounts of named tables are used and ERL_MAX_ETS_TABLES hasn't been increased, the performance of named table lookup will degrade.

Notice that there is no automatic garbage collection for tables. Even if there are no references to a table from any process, it is not automatically destroyed unless the owner process terminates. To destroy a table explicitly, use function delete/1. The default owner is the process that created the table. To transfer table ownership at process termination, use option heir or call give_away/3.

Some implementation details:

In the current implementation, every object insert and look-up operation results in a copy of the object.

'$end_of_table' is not to be used as a key, as this atom is used to mark the end of the table when using functions first/1 and next/2.

Notice the subtle difference between matching and comparing equal, which is demonstrated by table types set and ordered_set:

Two Erlang terms match if they are of the same type and have the same value, so that 1 matches 1, but not 1.0 (as 1.0 is a float/0 and not an integer/0).

Two Erlang terms compare equal if they either are of the same type and value, or if both are numeric types and extend to the same value, so that 1 compares equal to both 1 and 1.0.

The ordered_set works on the Erlang term order and no defined order exists between an integer/0 and a float/0 that extends to the same value. Hence the key 1 and the key 1.0 are regarded as equal in an ordered_set table.

Failures

Functions in this module fail by raising an error exception with error reason:

badarg - If any argument has the wrong format.

badarg - If the table identifier is invalid.

badarg - If the operation is denied because of table access rights (protected or private).

system_limit - Modification of a value causes it to not be representable internally in the VM. For example, incrementation of a counter past the largest integer representable.

system_limit - If a match specification passed as argument has excessive nesting which causes scheduler stack exhaustion for the scheduler that the calling process is executing on. Scheduler stack size can be configured when starting the runtime system.

Concurrency

This module provides some limited support for concurrent access. All updates to single objects are guaranteed to be both atomic and isolated. This means that an updating operation to a single object either succeeds or fails completely without any effect (atomicity) and that no intermediate results of the update can be seen by other processes (isolation). Some functions that update many objects state that they even guarantee atomicity and isolation for the entire operation. In database terms the isolation level can be seen as "serializable", as if all isolated operations are carried out serially, one after the other in a strict order.

Table traversal

There are different ways to traverse through the objects of a table.

Single-step traversal one key at at time, using first/1, next/2, last/1 and prev/2.

Single-step traversal one key at at time, but using first_lookup/1, next_lookup/2, last_lookup/1 and prev_lookup/2. This is more efficient when you also need to lookup the objects for the keys.

Search with simple match patterns, using match/1/2/3, match_delete/2 and match_object/1/2/3.

Search with more powerful match specifications, using select/1/2/3, select_count/2, select_delete/2, select_replace/2 and select_reverse/1/2/3.

Table conversions, using tab2file/2/3 and tab2list/1.

No table traversal will guarantee a consistent snapshot of the entire table if the table is also updated by concurrent processes during the traversal. The result of each concurrently updated object may be seen (or not) depending on if it has happened when the traversal visits that part of the table. The only way to guarantee a full consistent table snapshot (if you really need that) is to disallow concurrent updates during the entire traversal.

Moreover, traversals not done in a safe way, on tables where keys are inserted or deleted during the traversal, may yield the following undesired effects:

Any key may be missed.

Any key may be found more than once.

The traversal may fail with badarg exception if keys are deleted.

A table traversal is safe if either

the table is of type ordered_set.

the entire table traversal is done within one ETS function call.

function safe_fixtable/2 is used to keep the table fixated during the entire traversal.

Note

Even though the access of a single object is always guaranteed to be atomic and isolated, each traversal through a table to find the next key is not done with such guarantees. This is often not a problem, but may cause rare subtle "unexpected" effects if a concurrent process inserts objects during a traversal. For example, consider one process doing

ets:new(t, [ordered_set, named_table]),
ets:insert(t, {1}),
ets:insert(t, {2}),
ets:insert(t, {3}),

A concurrent call to ets:first(t), done by another process, may then in rare cases return 2 even though 2 has never existed in the table ordered as the first key. In the same way, a concurrent call to ets:next(t, 1) may return 3 even though 3 never existed in the table ordered directly after 1.

Effects like this are improbable but possible. The probability will further be reduced (if not vanish) if table option write_concurrency is not enabled. This can also only be a potential concern for ordered_set where the traversal order is defined.

Traversals using match and select functions may not need to scan the entire table depending on how the key is specified. A match pattern with a fully bound key (without any match variables) will optimize the operation to a single key lookup without any table traversal at all. For ordered_set a partially bound key will limit the traversal to only scan a subset of the table based on term order. A partially bound key is either a list or a tuple with a prefix that is fully bound. Example:

1> T = ets:new(t,[ordered_set]), ets:insert(T, {"555-1234", "John Smith"}).
true
2> %% Efficient search of all with area code 555
2> ets:match(T,{[$5,$5,$5,$- |'$1'],'$2'}).
[["1234","John Smith"]]

Match Specifications

Some of the functions use a match specification, match_spec. For a brief explanation, see select/2. For a detailed description, see section Match Specifications in Erlang in ERTS User's Guide.

A match specifications with excessive nesting will cause a system_limit error exception to be raised.

Summary

Types

comp_match_spec()

compiled_match_spec()

A compiled match specification.

continuation()

Opaque continuation used by select/1,3, select_reverse/1,3, match/1,3, and match_object/1,3.

match_pattern()

match_spec()

A match specification, see Match Specifications.

tab()

table()

table_access()

table_type()

tid()

A table identifier, as returned by new/2.

Functions

all()

Returns a list of all tables at the node. Named tables are specified by their names, unnamed tables are specified by their table identifiers.

delete(Table)

Deletes the entire table Table.

delete(Table, Key)

Deletes all objects with key Key from table Table. This function succeeds even if no objects with key Key exist.

delete_all_objects(Table)

Delete all objects in the ETS table Table. The operation is guaranteed to be atomic and isolated.

delete_object(Table, Object)

Delete the exact object Object from the ETS table, leaving objects with the same key but other differences (useful for type bag). In a duplicate_bag table, all instances of the object are deleted.

file2tab(Filename)

Reads a file produced by tab2file/2 or tab2file/3 and creates the corresponding table Table.

file2tab(Filename, Options)

Reads a file produced by tab2file/2 or tab2file/3 and creates the corresponding table Table.

first(Table)

Returns the first key Key in table Table. For an ordered_set table, the first key in Erlang term order is returned. For other table types, the first key according to the internal order of the table is returned. If the table is empty, '$end_of_table' is returned.

first_lookup(Table)

Similar to first/1 except that it returns the object(s) along with the key stored in the table. This is equivalent to doing first/1 followed by a lookup/2. If the table is empty, '$end_of_table' is returned.

foldl(Function, Acc0, Table)

Acc0 is returned if the table is empty. This function is similar to lists:foldl/3. The table elements are traversed in an unspecified order, except for ordered_set tables, where they are traversed first to last.

foldr(Function, Acc0, Table)

Acc0 is returned if the table is empty. This function is similar to lists:foldr/3. The table elements are traversed in an unspecified order, except for ordered_set tables, where they are traversed last to first.

from_dets(Table, DetsTab)

Fills an already created ETS table with the objects in the already opened Dets table DetsTab. Existing objects in the ETS table are kept unless overwritten.

fun2ms(LiteralFun)

Pseudo function that by a parse_transform translates LiteralFun typed as parameter in the function call to a match specification. With "literal" is meant that the fun must textually be written as the parameter of the function, it cannot be held in a variable that in turn is passed to the function.

give_away(Table, Pid, GiftData)

Make process Pid the new owner of table Table. If successful, message {'ETS-TRANSFER',Table,FromPid,GiftData} is sent to the new owner.

i()

Displays information about all ETS tables on a terminal.

i(Table)

Browses table Table on a terminal.

info(Table)

Returns information about table Table as a list of tuples. If Table has the correct type for a table identifier, but does not refer to an existing ETS table, undefined is returned. If Table is not of the correct type, a badarg exception is raised.

info(Table, Item)

Returns the information associated with Item for table Table, or returns undefined if Table does not refer an existing ETS table. If Table is not of the correct type, or if Item is not one of the allowed values, a badarg exception is raised.

init_table(Table, InitFun)

Replaces the existing objects of table Table with objects created by calling the input function InitFun, see below. This function is provided for compatibility with the dets module, it is not more efficient than filling a table by using insert/2.

insert(Table, ObjectOrObjects)

Inserts the object or all of the objects in list ObjectOrObjects into table Table.

insert_new(Table, ObjectOrObjects)

Same as insert/2 except that instead of overwriting objects with the same key (for set or ordered_set) or adding more objects with keys already existing in the table (for bag and duplicate_bag), false is returned.

is_compiled_ms(Term)

Checks if a term represent a valid compiled match specification. A compiled match specification is only valid on the Erlang node where it was compiled by calling match_spec_compile/1.

last(Table)

Returns the last key Key according to Erlang term order in table Table of type ordered_set. For other table types, the function is synonymous to first/1. If the table is empty, '$end_of_table' is returned.

last_lookup(Table)

Similar to last/1 except that it returns the object(s) along with the key stored in the table. This is equivalent to doing last/1 followed by a lookup/2. If the table is empty, '$end_of_table' is returned.

lookup(Table, Key)

Returns a list of all objects with key Key in table Table.

lookup_element(Table, Key, Pos)

For a table Table of type set or ordered_set, the function returns the Pos:th element of the object with key Key.

lookup_element(Table, Key, Pos, Default)

For a table Table of type set or ordered_set, the function returns the Pos:th element of the object with key Key.

match(Continuation)

Continues a match started with match/3. The next chunk of the size specified in the initial match/3 call is returned together with a new Continuation, which can be used in subsequent calls to this function.

match(Table, Pattern)

Matches the objects in table Table against pattern Pattern.

match(Table, Pattern, Limit)

Works like match/2, but returns only a limited (Limit) number of matching objects. Term Continuation can then be used in subsequent calls to match/1 to get the next chunk of matching objects. This is a space-efficient way to work on objects in a table, which is faster than traversing the table object by object using first/1 and next/2.

match_delete(Table, Pattern)

Deletes all objects that match pattern Pattern from table Table. For a description of patterns, see match/2.

match_object(Continuation)

Continues a match started with match_object/3. The next chunk of the size specified in the initial match_object/3 call is returned together with a new Continuation, which can be used in subsequent calls to this function.

match_object(Table, Pattern)

Matches the objects in table Table against pattern Pattern. For a description of patterns, see match/2. The function returns a list of all objects that match the pattern.

match_object(Table, Pattern, Limit)

Works like match_object/2, but only returns a limited (Limit) number of matching objects. Term Continuation can then be used in subsequent calls to match_object/1 to get the next chunk of matching objects. This is a space-efficient way to work on objects in a table, which is faster than traversing the table object by object using first/1 and next/2.

match_spec_compile(MatchSpec)

Transforms a match specification into an internal representation that can be used in subsequent calls to match_spec_run/2. The internal representation is opaque. To check the validity of a compiled match specification, use is_compiled_ms/1.

match_spec_run(List, CompiledMatchSpec)

Executes the matching specified in a compiled match specification on a list of terms. Term CompiledMatchSpec is to be the result of a call to match_spec_compile/1 and is hence the internal representation of the match specification one wants to use.

member(Table, Key)

Works like lookup/2, but does not return the objects. Returns true if one or more elements in the table has key Key, otherwise false.

new(Name, Options)

Creates a new table and returns a table identifier that can be used in subsequent operations. The table identifier can be sent to other processes so that a table can be shared between different processes within a node.

next(Table, Key1)

Returns the next key Key2, following key Key1 in table Table. For table type ordered_set, the next key in Erlang term order is returned. For other table types, the next key according to the internal order of the table is returned. If no next key exists, '$end_of_table' is returned.

next_lookup(Table, Key1)

Similar to next/2 except that it returns the object(s) along with the key stored in the table. This is equivalent to doing next/2 followed by a lookup/2. If no next key exists, '$end_of_table' is returned.

prev(Table, Key1)

Returns the previous key Key2, preceding key Key1 according to Erlang term order in table Table of type ordered_set. For other table types, the function is synonymous to next/2. If no previous key exists, '$end_of_table' is returned.

prev_lookup(Table, Key1)

Similar to prev/2 except that it returns the object(s) along with the key stored in the table. This is equivalent to doing prev/2 followed by a lookup/2. If no previous key exists, '$end_of_table' is returned.

rename(Table, Name)

Renames the named table Table to the new name Name. Afterwards, the old name cannot be used to access the table. Renaming an unnamed table has no effect.

repair_continuation(Continuation, MatchSpec)

Restores an opaque continuation returned by select/3 or select/1 if the continuation has passed through external term format (been sent between nodes or stored on disk).

safe_fixtable(Table, Fix)

Fixes a table of type set, bag, or duplicate_bag for safe traversal using first/1 & next/2, match/3 & match/1, match_object/3 & match_object/1, or select/3 & select/1.

select(Continuation)

Continues a match started with select/3. The next chunk of the size specified in the initial select/3 call is returned together with a new Continuation, which can be used in subsequent calls to this function.

select(Table, MatchSpec)

Matches the objects in table Table using a match specification. This is a more general call than match/2 and match_object/2 calls. In its simplest form, the match specification is as follows

select(Table, MatchSpec, Limit)

Works like select/2, but only returns a limited (Limit) number of matching objects. Term Continuation can then be used in subsequent calls to select/1 to get the next chunk of matching objects. This is a space-efficient way to work on objects in a table, which is still faster than traversing the table object by object using first/1 and next/2.

select_count(Table, MatchSpec)

Matches the objects in table Table using a match specification. If and only if the match specification returns true for an object, that object is considered a match and is counted. For any other result from the match specification the object is not considered a match and is therefore not counted.

select_delete(Table, MatchSpec)

Matches the objects in table Table using a match specification. If and only if the match specification returns true for an object, that object is removed from the table. For any other result from the match specification the object is retained. This is a more general function than match_delete/2.

select_replace(Table, MatchSpec)

Matches the objects in the table Table using a match specification. For each matched object, the existing object is replaced with the match specification result.

select_reverse(Continuation)

Continues a match started with select_reverse/3. For tables of type ordered_set, the traversal of the table continues to objects with keys earlier in the Erlang term order. The returned list also contains objects with keys in reverse order. For all other table types, the behavior is exactly that of select/1.

select_reverse(Table, MatchSpec)

Works like select/2, but returns the list in reverse order for table type ordered_set. For all other table types, the return value is identical to that of select/2.

select_reverse(Table, MatchSpec, Limit)

Works like select/3, but for table type ordered_set traversing is done starting at the last object in Erlang term order and moves to the first. For all other table types, the return value is identical to that of select/3.

setopts(Table, Opts)

Sets table options. The only allowed option to be set after the table has been created is heir. The calling process must be the table owner.

slot(Table, I)

This function is mostly for debugging purposes, normally first/next or last/prev are to be used instead.

tab2file(Table, Filename)

Dumps table Table to file Filename.

tab2file(Table, Filename, Options)

Dumps table Table to file Filename.

tab2list(Table)

Returns a list of all objects in table Table.

tabfile_info(Filename)

Returns information about the table dumped to file by tab2file/2 or tab2file/3.

table(Table)

Equivalent to table/2.

table(Table, Options)

Returns a Query List Comprehension (QLC) query handle. The qlc module provides a query language aimed mainly at Mnesia, but ETS tables, Dets tables, and lists are also recognized by QLC as sources of data. Calling table/1,2 is the means to make the ETS table Table usable to QLC.

take(Table, Key)

Returns and removes a list of all objects with key Key in table Table.

test_ms(Tuple, MatchSpec)

This function is a utility to test a match specification used in calls to select/2. The function both tests MatchSpec for "syntactic" correctness and runs the match specification against object Tuple.

to_dets(Table, DetsTab)

Fills an already created/opened Dets table with the objects in the already opened ETS table named Table. The Dets table is emptied before the objects are inserted.

update_counter/3

Equivalent to update_counter/4.

update_counter/4

This function provides an efficient way to update one or more counters, without the trouble of having to look up an object, update the object by incrementing an element, and insert the resulting object into the table again. The operation is guaranteed to be atomic and isolated.

update_element(Table, Key, ElementSpec)

Equivalent to update_element/4.

update_element(Table, Key, ElementSpec, Default)

This function provides an efficient way to update one or more elements within an object, without the trouble of having to look up, update, and write back the entire object.

whereis(TableName)

This function returns the tid/0 of the named table identified by TableName, or undefined if no such table exists. The tid/0 can be used in place of the table name in all operations, which is slightly faster since the name does not have to be resolved on each call.

Types

comp_match_spec()

-type comp_match_spec() :: compiled_match_spec().

compiled_match_spec()

-opaque compiled_match_spec()

A compiled match specification.

continuation()

(not exported)

-type continuation() ::
          '$end_of_table' |
          {table(), integer(), integer(), compiled_match_spec(), list(), integer()} |
          {table(), _, _, integer(), compiled_match_spec(), list(), integer(), integer()}.

Opaque continuation used by select/1,3, select_reverse/1,3, match/1,3, and match_object/1,3.

match_pattern()

-type match_pattern() :: atom() | tuple().

match_spec()

-type match_spec() :: [{match_pattern(), [_], [_]}].

A match specification, see Match Specifications.

tab()

-type tab() :: table().

table()

-type table() :: atom() | tid().

table_access()

-type table_access() :: public | protected | private.

table_type()

-type table_type() :: set | ordered_set | bag | duplicate_bag.

tid()

-opaque tid()

A table identifier, as returned by new/2.

Functions

all()

-spec all() -> [Table] when Table :: table().

Returns a list of all tables at the node. Named tables are specified by their names, unnamed tables are specified by their table identifiers.

There is no guarantee of consistency in the returned list. Tables created or deleted by other processes "during" the ets:all() call either are or are not included in the list. Only tables created/deleted before ets:all() is called are guaranteed to be included/excluded.

delete(Table)

-spec delete(Table) -> true when Table :: table().

Deletes the entire table Table.

delete(Table, Key)

-spec delete(Table, Key) -> true when Table :: table(), Key :: term().

Deletes all objects with key Key from table Table. This function succeeds even if no objects with key Key exist.

delete_all_objects(Table)

-spec delete_all_objects(Table) -> true when Table :: table().

Delete all objects in the ETS table Table. The operation is guaranteed to be atomic and isolated.

delete_object(Table, Object)

-spec delete_object(Table, Object) -> true when Table :: table(), Object :: tuple().

Delete the exact object Object from the ETS table, leaving objects with the same key but other differences (useful for type bag). In a duplicate_bag table, all instances of the object are deleted.

file2tab(Filename)

-spec file2tab(Filename) -> {ok, Table} | {error, Reason}
                  when Filename :: file:name(), Table :: table(), Reason :: term().

Reads a file produced by tab2file/2 or tab2file/3 and creates the corresponding table Table.

Equivalent to file2tab(Filename, []).

file2tab(Filename, Options)

-spec file2tab(Filename, Options) -> {ok, Table} | {error, Reason}
                  when
                      Filename :: file:name(),
                      Table :: table(),
                      Options :: [Option],
                      Option :: {verify, boolean()},
                      Reason :: term().

Reads a file produced by tab2file/2 or tab2file/3 and creates the corresponding table Table.

The only supported option is {verify,boolean()}. If verification is turned on (by specifying {verify,true}), the function uses whatever information is present in the file to assert that the information is not damaged. How this is done depends on which extended_info was written using tab2file/3.

If no extended_info is present in the file and {verify,true} is specified, the number of objects written is compared to the size of the original table when the dump was started. This can make verification fail if the table was public and objects were added or removed while the table was dumped to file. To avoid this problem, either do not verify files dumped while updated simultaneously or use option {extended_info, [object_count]} to tab2file/3, which extends the information in the file with the number of objects written.

If verification is turned on and the file was written with option {extended_info, [md5sum]}, reading the file is slower and consumes radically more CPU time than otherwise.

{verify,false} is the default.

first(Table)

-spec first(Table) -> Key | '$end_of_table' when Table :: table(), Key :: term().

Returns the first key Key in table Table. For an ordered_set table, the first key in Erlang term order is returned. For other table types, the first key according to the internal order of the table is returned. If the table is empty, '$end_of_table' is returned.

To find subsequent keys in the table, use next/2.

first_lookup(Table)

(since OTP 27.0)

-spec first_lookup(Table) -> {Key, [Object]} | '$end_of_table'
                      when Table :: table(), Key :: term(), Object :: tuple().

Similar to first/1 except that it returns the object(s) along with the key stored in the table. This is equivalent to doing first/1 followed by a lookup/2. If the table is empty, '$end_of_table' is returned.

To find subsequent objects in the table, use next_lookup/2.

foldl(Function, Acc0, Table)

-spec foldl(Function, Acc0, Table) -> Acc1
               when
                   Function :: fun((Element :: term(), AccIn) -> AccOut),
                   Table :: table(),
                   Acc0 :: term(),
                   Acc1 :: term(),
                   AccIn :: term(),
                   AccOut :: term().

Acc0 is returned if the table is empty. This function is similar to lists:foldl/3. The table elements are traversed in an unspecified order, except for ordered_set tables, where they are traversed first to last.

If Function inserts objects into the table, or another process inserts objects into the table, those objects can (depending on key ordering) be included in the traversal.

foldr(Function, Acc0, Table)

-spec foldr(Function, Acc0, Table) -> Acc1
               when
                   Function :: fun((Element :: term(), AccIn) -> AccOut),
                   Table :: table(),
                   Acc0 :: term(),
                   Acc1 :: term(),
                   AccIn :: term(),
                   AccOut :: term().

Acc0 is returned if the table is empty. This function is similar to lists:foldr/3. The table elements are traversed in an unspecified order, except for ordered_set tables, where they are traversed last to first.

If Function inserts objects into the table, or another process inserts objects into the table, those objects can (depending on key ordering) be included in the traversal.

from_dets(Table, DetsTab)

-spec from_dets(Table, DetsTab) -> true when Table :: table(), DetsTab :: dets:tab_name().

Fills an already created ETS table with the objects in the already opened Dets table DetsTab. Existing objects in the ETS table are kept unless overwritten.

If any of the tables does not exist or the Dets table is not open, a badarg exception is raised.

fun2ms(LiteralFun)

-spec fun2ms(LiteralFun) -> MatchSpec when LiteralFun :: function(), MatchSpec :: match_spec().

Pseudo function that by a parse_transform translates LiteralFun typed as parameter in the function call to a match specification. With "literal" is meant that the fun must textually be written as the parameter of the function, it cannot be held in a variable that in turn is passed to the function.

The parse transform is provided in the ms_transform module and the source must include file ms_transform.hrl in STDLIB for this pseudo function to work. Failing to include the hrl file in the source results in a runtime error, not a compile time error. The include file is easiest included by adding line -include_lib("stdlib/include/ms_transform.hrl"). to the source file.

The fun is very restricted, it can take only a single parameter (the object to match): a sole variable or a tuple. It must use the is_ guard tests. Language constructs that have no representation in a match specification (if, case, receive, and so on) are not allowed.

The return value is the resulting match specification.

Example:

1> ets:fun2ms(fun({M,N}) when N > 3 -> M end).
[{{'$1','$2'},[{'>','$2',3}],['$1']}]

Variables from the environment can be imported, so that the following works:

2> X=3.
3
3> ets:fun2ms(fun({M,N}) when N > X -> M end).
[{{'$1','$2'},[{'>','$2',{const,3}}],['$1']}]

The imported variables are replaced by match specification const expressions, which is consistent with the static scoping for Erlang funs. However, local or global function calls cannot be in the guard or body of the fun. Calls to built-in match specification functions is of course allowed:

4> ets:fun2ms(fun({M,N}) when N > X, my_fun(M) -> M end).
Error: fun containing local Erlang function calls
('my_fun' called in guard) cannot be translated into match_spec
{error,transform_error}
5> ets:fun2ms(fun({M,N}) when N > X, is_atom(M) -> M end).
[{{'$1','$2'},[{'>','$2',{const,3}},{is_atom,'$1'}],['$1']}]

As shown by the example, the function can be called from the shell also. The fun must be literally in the call when used from the shell as well.

Warning

If the parse_transform is not applied to a module that calls this pseudo function, the call fails in runtime (with a badarg). The ets module exports a function with this name, but it is never to be called except when using the function in the shell. If the parse_transform is properly applied by including header file ms_transform.hrl, compiled code never calls the function, but the function call is replaced by a literal match specification.

For more information, see ms_transform.

give_away(Table, Pid, GiftData)

-spec give_away(Table, Pid, GiftData) -> true when Table :: table(), Pid :: pid(), GiftData :: term().

Make process Pid the new owner of table Table. If successful, message {'ETS-TRANSFER',Table,FromPid,GiftData} is sent to the new owner.

The process Pid must be alive, local, and not already the owner of the table. The calling process must be the table owner.

Notice that this function does not affect option heir of the table. A table owner can, for example, set heir to itself, give the table away, and then get it back if the receiver terminates.

i()

-spec i() -> ok.

Displays information about all ETS tables on a terminal.

i(Table)

-spec i(Table) -> ok when Table :: table().

Browses table Table on a terminal.

info(Table)

-spec info(Table) -> InfoList | undefined
              when
                  Table :: table(),
                  InfoList :: [InfoTuple],
                  InfoTuple ::
                      {compressed, boolean()} |
                      {decentralized_counters, boolean()} |
                      {heir, pid() | none} |
                      {id, tid()} |
                      {keypos, pos_integer()} |
                      {memory, non_neg_integer()} |
                      {name, atom()} |
                      {named_table, boolean()} |
                      {node, node()} |
                      {owner, pid()} |
                      {protection, table_access()} |
                      {size, non_neg_integer()} |
                      {type, table_type()} |
                      {write_concurrency, boolean() | auto} |
                      {read_concurrency, boolean()}.

Returns information about table Table as a list of tuples. If Table has the correct type for a table identifier, but does not refer to an existing ETS table, undefined is returned. If Table is not of the correct type, a badarg exception is raised.

{compressed, boolean()} - Indicates if the table is compressed.

{decentralized_counters, boolean()} - Indicates whether the table uses decentralized_counters.

{heir, pid() | none} - The pid of the heir of the table, or none if no heir is set.

{id, tid()} - The table identifier.

{keypos, integer() >= 1} - The key position.

{memory, integer() >= 0} - The number of words allocated to the table.

{name, atom()} - The table name.

{named_table, boolean()} - Indicates if the table is named.

{node, node()} - The node where the table is stored. This field is no longer meaningful, as tables cannot be accessed from other nodes.

{owner, pid()} - The pid of the owner of the table.

{protection, access()} - The table access rights.

{size, integer() >= 0} - The number of objects inserted in the table.

{type, type()} - The table type.

{read_concurrency, boolean()} - Indicates whether the table uses read_concurrency or not.

{write_concurrency, WriteConcurrencyAlternative} - Indicates which write_concurrency option the table uses.

Note

The execution time of this function is affected by the decentralized_counters table option. The execution time is much longer when the decentralized_counters option is set to true than when the decentralized_counters option is set to false.

info(Table, Item)

-spec info(Table, Item) -> Value | undefined
              when
                  Table :: table(),
                  Item ::
                      binary | compressed | decentralized_counters | fixed | heir | id | keypos |
                      memory | name | named_table | node | owner | protection | safe_fixed |
                      safe_fixed_monotonic_time | size | stats | type | write_concurrency |
                      read_concurrency,
                  Value :: term().

Returns the information associated with Item for table Table, or returns undefined if Table does not refer an existing ETS table. If Table is not of the correct type, or if Item is not one of the allowed values, a badarg exception is raised.

In addition to the {Item,Value} pairs defined for info/1, the following items are allowed:

Item=binary, Value=BinInfo

BinInfo is a list containing miscellaneous information about binaries kept by the table. This Item can be changed or removed without prior notice. In the current implementation BinInfo is a list of tuples {BinaryId,BinarySize,BinaryRefcCount}.

Item=fixed, Value=boolean()

Indicates if the table is fixed by any process.

Item=safe_fixed|safe_fixed_monotonic_time, Value={FixationTime,Info}|false

If the table is fixed using safe_fixtable/2, the call returns a tuple where FixationTime is the last time when the table changed from unfixed to fixed.

The format and value of FixationTime depends on Item:

safe_fixed - FixationTime corresponds to the result returned by erlang:timestamp/0 at the time of fixation. Notice that when the system uses single or multi time warp modes this can produce strange results, as the use of safe_fixed is not time warp safe. Time warp safe code must use safe_fixed_monotonic_time instead.

safe_fixed_monotonic_time - FixationTime corresponds to the result returned by erlang:monotonic_time/0 at the time of fixation. The use of safe_fixed_monotonic_time is time warp safe.

Info is a possibly empty lists of tuples {Pid,RefCount}, one tuple for every process the table is fixed by now. RefCount is the value of the reference counter and it keeps track of how many times the table has been fixed by the process.

Table fixations are not limited to safe_fixtable/2. Temporary fixations may also be done by for example traversing functions like select and match. Such table fixations are automatically released before the corresponding functions returns, but they may be seen by a concurrent call to ets:info(T,safe_fixed|safe_fixed_monotonic_time).

If the table is not fixed at all, the call returns false.

Item=stats, Value=tuple()

Returns internal statistics about tables on an internal format used by OTP test suites. Not for production use.

Note

The execution time of this function is affected by the decentralized_counters table option when the second argument of the function is size or memory. The execution time is much longer when the decentralized_counters option is set to true than when the decentralized_counters option is set to false.

init_table(Table, InitFun)

-spec init_table(Table, InitFun) -> true
                    when
                        Table :: table(),
                        InitFun :: fun((Arg) -> Res),
                        Arg :: read | close,
                        Res :: end_of_input | {Objects :: [term()], InitFun} | term().

Replaces the existing objects of table Table with objects created by calling the input function InitFun, see below. This function is provided for compatibility with the dets module, it is not more efficient than filling a table by using insert/2.

When called with argument read, the function InitFun is assumed to return end_of_input when there is no more input, or {Objects, Fun}, where Objects is a list of objects and Fun is a new input function. Any other value Value is returned as an error {error, {init_fun, Value}}. Each input function is called exactly once, and if an error occur, the last function is called with argument close, the reply of which is ignored.

If the table type is set and more than one object exists with a given key, one of the objects is chosen. This is not necessarily the last object with the given key in the sequence of objects returned by the input functions. This holds also for duplicated objects stored in tables of type bag.

insert(Table, ObjectOrObjects)

-spec insert(Table, ObjectOrObjects) -> true
                when Table :: table(), ObjectOrObjects :: tuple() | [tuple()].

Inserts the object or all of the objects in list ObjectOrObjects into table Table.

If the table type is set and the key of the inserted objects matches the key of any object in the table, the old object is replaced.

If the table type is ordered_set and the key of the inserted object compares equal to the key of any object in the table, the old object is replaced.

If the table type is bag and the object matches any whole object in the table, the object is not inserted.

If the list contains more than one object with matching keys and the table type is set, one is inserted, which one is not defined. The same holds for table type ordered_set if the keys compare equal.

The entire operation is guaranteed to be atomic and isolated, even when a list of objects is inserted.

For bag and duplicate_bag, objects in the list with identical keys will be inserted in list order (from head to tail). That is, a subsequent call to lookup(T,Key) will return them in that inserted order.

Note

For bag the insertion order of indentical keys described above was accidentally reverted in OTP 23.0 and later fixed in OTP 25.3. That is, from OTP 23.0 up until OTP 25.3 the objects in a list are inserted in reverse order (from tail to head).

For duplicate_bag the same faulty reverse insertion exist from OTP 23.0 until OTP 25.3. However, it is unpredictable and may or may not happen. A longer list will increase the probabiliy of the insertion being done in reverse.

insert_new(Table, ObjectOrObjects)

-spec insert_new(Table, ObjectOrObjects) -> boolean()
                    when Table :: table(), ObjectOrObjects :: tuple() | [tuple()].

Same as insert/2 except that instead of overwriting objects with the same key (for set or ordered_set) or adding more objects with keys already existing in the table (for bag and duplicate_bag), false is returned.

If ObjectOrObjects is a list, the function checks every key before inserting anything. Nothing is inserted unless all keys present in the list are absent from the table. Like insert/2, the entire operation is guaranteed to be atomic and isolated.

is_compiled_ms(Term)

-spec is_compiled_ms(Term) -> boolean() when Term :: term().

Checks if a term represent a valid compiled match specification. A compiled match specification is only valid on the Erlang node where it was compiled by calling match_spec_compile/1.

Note

Before STDLIB 3.4 (OTP 20.0) compiled match specifications did not have an external representation. If passed through binary_to_term(term_to_binary(CMS)) or sent to another node and back, the result was always an empty binary <<>>.

After STDLIB 3.4 (OTP 20.0) compiled match specifications have an external representation as a node specific reference to the original compiled match specification. If passed through binary_to_term(term_to_binary(CMS)) or sent to another node and back, the result may or may not be a valid compiled match specification depending on if the original compiled match specification was still alive.

last(Table)

-spec last(Table) -> Key | '$end_of_table' when Table :: table(), Key :: term().

Returns the last key Key according to Erlang term order in table Table of type ordered_set. For other table types, the function is synonymous to first/1. If the table is empty, '$end_of_table' is returned.

To find preceding keys in the table, use prev/2.

last_lookup(Table)

(since OTP 27.0)

-spec last_lookup(Table) -> {Key, [Object]} | '$end_of_table'
                     when Table :: table(), Key :: term(), Object :: tuple().

Similar to last/1 except that it returns the object(s) along with the key stored in the table. This is equivalent to doing last/1 followed by a lookup/2. If the table is empty, '$end_of_table' is returned.

To find preceding objects in the table, use prev_lookup/2.

lookup(Table, Key)

-spec lookup(Table, Key) -> [Object] when Table :: table(), Key :: term(), Object :: tuple().

Returns a list of all objects with key Key in table Table.

For tables of type set, bag, or duplicate_bag, an object is returned only if the specified key matches the key of the object in the table.

For tables of type ordered_set, an object is returned if the specified key compares equal to the key of an object in the table.

The difference is the same as between =:= and ==.

As an example, one can insert an object with integer/0 1 as a key in an ordered_set and get the object returned as a result of doing a lookup/2 with float/0 1.0 as the key to search for.

For tables of type set or ordered_set, the function returns either the empty list or a list with one element, as there cannot be more than one object with the same key. For tables of type bag or duplicate_bag, the function returns a list of arbitrary length.

Notice that the sequential order of object insertions is preserved; the first object inserted with the specified key is the first in the resulting list, and so on. See also the note about list insertion order.

lookup_element(Table, Key, Pos)

-spec lookup_element(Table, Key, Pos) -> Elem
                        when
                            Table :: table(),
                            Key :: term(),
                            Pos :: pos_integer(),
                            Elem :: term() | [term()].

For a table Table of type set or ordered_set, the function returns the Pos:th element of the object with key Key.

For tables of type bag or duplicate_bag, the functions returns a list with the Pos:th element of every object with key Key.

If no object with key Key exists, the function exits with reason badarg.

If Pos is larger than the size of any tuple with a matching key, the function exits with reason badarg.

The difference between set, bag, and duplicate_bag on one hand, and ordered_set on the other, regarding the fact that ordered_set view keys as equal when they compare equal whereas the other table types regard them equal only when they match, holds for lookup_element/3.

lookup_element(Table, Key, Pos, Default)

(since OTP 26.0)

-spec lookup_element(Table, Key, Pos, Default) -> Elem | Default
                        when
                            Table :: table(),
                            Key :: term(),
                            Pos :: pos_integer(),
                            Default :: term(),
                            Elem :: term() | [term()].

For a table Table of type set or ordered_set, the function returns the Pos:th element of the object with key Key.

For tables of type bag or duplicate_bag, the functions returns a list with the Pos:th element of every object with key Key.

If no object with key Key exists, the function returns Default.

If Pos is larger than the size of any tuple with a matching key, the function exits with reason badarg.

The difference between set, bag, and duplicate_bag on one hand, and ordered_set on the other, regarding the fact that ordered_set view keys as equal when they compare equal whereas the other table types regard them equal only when they match, holds for lookup_element/4.

match(Continuation)

-spec match(Continuation) -> {[Match], Continuation} | '$end_of_table'
               when Match :: [term()], Continuation :: continuation().

Continues a match started with match/3. The next chunk of the size specified in the initial match/3 call is returned together with a new Continuation, which can be used in subsequent calls to this function.

When there are no more objects in the table, '$end_of_table' is returned.

match(Table, Pattern)

-spec match(Table, Pattern) -> [Match]
               when Table :: table(), Pattern :: match_pattern(), Match :: [term()].

Matches the objects in table Table against pattern Pattern.

A pattern is a term that can contain:

Bound parts (Erlang terms)

'_' that matches any Erlang term

Pattern variables '$N', where N=0,1,...

The function returns a list with one element for each matching object, where each element is an ordered list of pattern variable bindings, for example:

6> ets:match(T, '$1'). % Matches every object in table
[[{rufsen,dog,7}],[{brunte,horse,5}],[{ludde,dog,5}]]
7> ets:match(T, {'_',dog,'$1'}).
[[7],[5]]
8> ets:match(T, {'_',cow,'$1'}).
[]

If the key is specified in the pattern, the match is very efficient. If the key is not specified, that is, if it is a variable or an underscore, the entire table must be searched. The search time can be substantial if the table is very large.

For tables of type ordered_set, the result is in the same order as in a first/next traversal.

match(Table, Pattern, Limit)

-spec match(Table, Pattern, Limit) -> {[Match], Continuation} | '$end_of_table'
               when
                   Table :: table(),
                   Pattern :: match_pattern(),
                   Limit :: pos_integer(),
                   Match :: [term()],
                   Continuation :: continuation().

Works like match/2, but returns only a limited (Limit) number of matching objects. Term Continuation can then be used in subsequent calls to match/1 to get the next chunk of matching objects. This is a space-efficient way to work on objects in a table, which is faster than traversing the table object by object using first/1 and next/2.

If the table is empty, '$end_of_table' is returned.

Use safe_fixtable/2 to guarantee safe traversal for subsequent calls to match/1.

match_delete(Table, Pattern)

-spec match_delete(Table, Pattern) -> true when Table :: table(), Pattern :: match_pattern().

Deletes all objects that match pattern Pattern from table Table. For a description of patterns, see match/2.

match_object(Continuation)

-spec match_object(Continuation) -> {[Object], Continuation} | '$end_of_table'
                      when Object :: tuple(), Continuation :: continuation().

Continues a match started with match_object/3. The next chunk of the size specified in the initial match_object/3 call is returned together with a new Continuation, which can be used in subsequent calls to this function.

When there are no more objects in the table, '$end_of_table' is returned.

match_object(Table, Pattern)

-spec match_object(Table, Pattern) -> [Object]
                      when Table :: table(), Pattern :: match_pattern(), Object :: tuple().

Matches the objects in table Table against pattern Pattern. For a description of patterns, see match/2. The function returns a list of all objects that match the pattern.

If the key is specified in the pattern, the match is very efficient. If the key is not specified, that is, if it is a variable or an underscore, the entire table must be searched. The search time can be substantial if the table is very large.

For tables of type ordered_set, the result is in the same order as in a first/next traversal.

match_object(Table, Pattern, Limit)

-spec match_object(Table, Pattern, Limit) -> {[Object], Continuation} | '$end_of_table'
                      when
                          Table :: table(),
                          Pattern :: match_pattern(),
                          Limit :: pos_integer(),
                          Object :: tuple(),
                          Continuation :: continuation().

Works like match_object/2, but only returns a limited (Limit) number of matching objects. Term Continuation can then be used in subsequent calls to match_object/1 to get the next chunk of matching objects. This is a space-efficient way to work on objects in a table, which is faster than traversing the table object by object using first/1 and next/2.

If the table is empty, '$end_of_table' is returned.

Use safe_fixtable/2 to guarantee safe traversal for subsequent calls to match_object/1.

match_spec_compile(MatchSpec)

-spec match_spec_compile(MatchSpec) -> CompiledMatchSpec
                            when MatchSpec :: match_spec(), CompiledMatchSpec :: compiled_match_spec().

Transforms a match specification into an internal representation that can be used in subsequent calls to match_spec_run/2. The internal representation is opaque. To check the validity of a compiled match specification, use is_compiled_ms/1.

If term MatchSpec does not represent a valid match specification, a badarg exception is raised.

Note

This function has limited use in normal code. It is used by the dets module to perform the dets:select/1 operations.

match_spec_run(List, CompiledMatchSpec)

-spec match_spec_run(List, CompiledMatchSpec) -> list()
                        when List :: [term()], CompiledMatchSpec :: compiled_match_spec().

Executes the matching specified in a compiled match specification on a list of terms. Term CompiledMatchSpec is to be the result of a call to match_spec_compile/1 and is hence the internal representation of the match specification one wants to use.

The matching is executed on each element in List and the function returns a list containing all results. If an element in List does not match, nothing is returned for that element. The length of the result list is therefore equal or less than the length of parameter List.

Example:

The following two calls give the same result (but certainly not the same execution time):

Table = ets:new...
MatchSpec = ...
% The following call...
ets:match_spec_run(ets:tab2list(Table),
                   ets:match_spec_compile(MatchSpec)),
% ...gives the same result as the more common (and more efficient)
ets:select(Table, MatchSpec),

Note

This function has limited use in normal code. It is used by the dets module to perform the dets:select/1 operations and by Mnesia during transactions.

member(Table, Key)

-spec member(Table, Key) -> boolean() when Table :: table(), Key :: term().

Works like lookup/2, but does not return the objects. Returns true if one or more elements in the table has key Key, otherwise false.

new(Name, Options)

-spec new(Name, Options) -> table()
             when
                 Name :: atom(),
                 Options :: [Option],
                 Option ::
                     Type | Access | named_table |
                     {keypos, Pos} |
                     {heir, Pid} |
                     {heir, Pid, HeirData} |
                     {heir, none} |
                     Tweaks,
                 Type :: table_type(),
                 Access :: table_access(),
                 WriteConcurrencyAlternative :: boolean() | auto,
                 Tweaks ::
                     {write_concurrency, WriteConcurrencyAlternative} |
                     {read_concurrency, boolean()} |
                     {decentralized_counters, boolean()} |
                     compressed,
                 Pos :: pos_integer(),
                 Pid :: pid(),
                 HeirData :: term().

Creates a new table and returns a table identifier that can be used in subsequent operations. The table identifier can be sent to other processes so that a table can be shared between different processes within a node.

Parameter Options is a list of options that specifies table type, access rights, key position, and whether the table is named. Default values are used for omitted options. This means that not specifying any options ([]) is the same as specifying [set, protected, {keypos,1}, {heir,none}, {write_concurrency,false}, {read_concurrency,false}, {decentralized_counters,false}].

set - The table is a set table: one key, one object, no order among objects. This is the default table type.

ordered_set - The table is a ordered_set table: one key, one object, ordered in Erlang term order, which is the order implied by the < and > operators. Tables of this type have a somewhat different behavior in some situations than tables of other types. Most notably, the ordered_set tables regard keys as equal when they compare equal, not only when they match. This means that to an ordered_set table, integer/0 1 and float/0 1.0 are regarded as equal. This also means that the key used to lookup an element does not necessarily match the key in the returned elements, if float/0's and integer/0's are mixed in keys of a table.

bag - The table is a bag table, which can have many objects, but only one instance of each object, per key.

duplicate_bag - The table is a duplicate_bag table, which can have many objects, including multiple copies of the same object, per key.

public - Any process can read or write to the table.

protected - The owner process can read and write to the table. Other processes can only read the table. This is the default setting for the access rights.

private - Only the owner process can read or write to the table.

named_table - If this option is present, the table is registered under its Name which can then be used instead of the table identifier in subsequent operations.

The function will also return the Name instead of the table identifier. To get the table identifier of a named table, use whereis/1.

{keypos,Pos} - Specifies which element in the stored tuples to use as key. By default, it is the first element, that is, Pos=1. However, this is not always appropriate. In particular, we do not want the first element to be the key if we want to store Erlang records in a table.

Notice that any tuple stored in the table must have at least Pos number of elements.

{heir,Pid,HeirData} | {heir,Pid} | {heir,none} - Set a process as heir. The heir inherits the table if the owner terminates. If HeirData is given, a message {'ETS-TRANSFER',tid(),FromPid,HeirData} is sent to the heir when that occurs. If {heir,Pid} is given, no 'ETS-TRANSFER' message is sent. The user must then make sure the heir gets notified some other way (through a link or monitor for example) to avoid the table being left unnoticed by its new owner.

The heir must be a local process. Default heir is none, which destroys the table when the owner terminates.

{write_concurrency,WriteConcurrencyAlternative} - Performance tuning. Defaults to false, in which case an operation that mutates (writes to) the table obtains exclusive access, blocking any concurrent access of the same table until finished. If set to true, the table is optimized for concurrent write access. Different objects of the same table can be mutated (and read) by concurrent processes. This is achieved to some degree at the expense of memory consumption and the performance of sequential access and concurrent reading.

The auto alternative for the write_concurrency option is similar to the true option but automatically adjusts the synchronization granularity during runtime depending on how the table is used. This is the recommended write_concurrency option when using Erlang/OTP 25 and above as it performs well in most scenarios.

The write_concurrency option can be combined with the options read_concurrency and decentralized_counters. You typically want to combine write_concurrency with read_concurrency when large concurrent read bursts and large concurrent write bursts are common; for more information, see option read_concurrency. It is almost always a good idea to combine the write_concurrency option with the decentralized_counters option.

Notice that this option does not change any guarantees about atomicity and isolation. Functions that makes such promises over many objects (like insert/2) gain less (or nothing) from this option.

The memory consumption inflicted by both write_concurrency and read_concurrency is a constant overhead per table for set, bag and duplicate_bag when the true alternative for the write_concurrency option is not used. For all tables with the auto alternative and ordered_set tables with true alternative the memory overhead depends on the amount of actual detected concurrency during runtime. The memory overhead can be especially large when both write_concurrency and read_concurrency are combined.

Note

Prior to stdlib-3.7 (OTP-22.0) write_concurrency had no effect on ordered_set.

Note

The auto alternative for the write_concurrency option is only available in OTP-25.0 and above.

{read_concurrency,boolean()}(Since OTP R14B)
Performance tuning. Defaults to false. When set to true, the table is optimized for concurrent read operations. When this option is enabled read operations become much cheaper; especially on systems with multiple physical processors. However, switching between read and write operations becomes more expensive.

You typically want to enable this option when concurrent read operations are much more frequent than write operations, or when concurrent reads and writes comes in large read and write bursts (that is, many reads not interrupted by writes, and many writes not interrupted by reads).

You typically do not want to enable this option when the common access pattern is a few read operations interleaved with a few write operations repeatedly. In this case, you would get a performance degradation by enabling this option.

Option read_concurrency can be combined with option write_concurrency. You typically want to combine these when large concurrent read bursts and large concurrent write bursts are common.

{decentralized_counters,boolean()}(Since OTP 23.0)
Performance tuning. Defaults to true for all tables with the write_concurrency option set to auto. For tables of type ordered_set the option also defaults to true when the write_concurrency option is set to true. The option defaults to false for all other configurations. This option has no effect if the write_concurrency option is set to false.

When this option is set to true, the table is optimized for frequent concurrent calls to operations that modify the tables size and/or its memory consumption (e.g., insert/2 and delete/2). The drawback is that calls to info/1 and info/2 with size or memory as the second argument can get much slower when the decentralized_counters option is turned on.

When this option is enabled the counters for the table size and memory consumption are distributed over several cache lines and the scheduling threads are mapped to one of those cache lines. The erl option +dcg can be used to control the number of cache lines that the counters are distributed over.

compressed(Since OTP R14B01)
If this option is present, the table data is stored in a more compact format to consume less memory. However, it will make table operations slower. Especially operations that need to inspect entire objects, such as match and select, get much slower. The key element is not compressed.

next(Table, Key1)

-spec next(Table, Key1) -> Key2 | '$end_of_table' when Table :: table(), Key1 :: term(), Key2 :: term().

Returns the next key Key2, following key Key1 in table Table. For table type ordered_set, the next key in Erlang term order is returned. For other table types, the next key according to the internal order of the table is returned. If no next key exists, '$end_of_table' is returned.

To find the first key in the table, use first/1.

Unless a table of type set, bag, or duplicate_bag is fixated using safe_fixtable/2, a call to next/2 will fail if Key1 no longer exists in the table. For table type ordered_set, the function always returns the next key after Key1 in term order, regardless whether Key1 ever existed in the table.

next_lookup(Table, Key1)

(since OTP 27.0)

-spec next_lookup(Table, Key1) -> {Key2, [Object]} | '$end_of_table'
                     when Table :: table(), Key1 :: term(), Key2 :: term(), Object :: tuple().

Similar to next/2 except that it returns the object(s) along with the key stored in the table. This is equivalent to doing next/2 followed by a lookup/2. If no next key exists, '$end_of_table' is returned.

It can be interleaved with next/2 during traversal.

prev(Table, Key1)

-spec prev(Table, Key1) -> Key2 | '$end_of_table' when Table :: table(), Key1 :: term(), Key2 :: term().

Returns the previous key Key2, preceding key Key1 according to Erlang term order in table Table of type ordered_set. For other table types, the function is synonymous to next/2. If no previous key exists, '$end_of_table' is returned.

To find the last key in an ordered_set table, use last/1.

prev_lookup(Table, Key1)

(since OTP 27.0)

-spec prev_lookup(Table, Key1) -> {Key2, [Object]} | '$end_of_table'
                     when Table :: table(), Key1 :: term(), Key2 :: term(), Object :: tuple().

Similar to prev/2 except that it returns the object(s) along with the key stored in the table. This is equivalent to doing prev/2 followed by a lookup/2. If no previous key exists, '$end_of_table' is returned.

It can be interleaved with prev/2 during traversal.

rename(Table, Name)

-spec rename(Table, Name) -> Name when Table :: table(), Name :: atom().

Renames the named table Table to the new name Name. Afterwards, the old name cannot be used to access the table. Renaming an unnamed table has no effect.

repair_continuation(Continuation, MatchSpec)

-spec repair_continuation(Continuation, MatchSpec) -> Continuation
                             when Continuation :: continuation(), MatchSpec :: match_spec().

Restores an opaque continuation returned by select/3 or select/1 if the continuation has passed through external term format (been sent between nodes or stored on disk).

The reason for this function is that continuation terms contain compiled match specifications and may therefore be invalidated if converted to external term format. Given that the original match specification is kept intact, the continuation can be restored, meaning it can once again be used in subsequent select/1 calls even though it has been stored on disk or on another node.

Examples:

The following sequence of calls may fail:

T=ets:new(x,[]),
...
MS = ets:fun2ms(fun({N,_}=A) when (N rem 10) =:= 0 -> A end),
{_,C} = ets:select(T, MS, 10),
MaybeBroken = binary_to_term(term_to_binary(C)),
ets:select(MaybeBroken).

The following sequence works, as the call to repair_continuation/2 reestablishes the MaybeBroken continuation.

T=ets:new(x,[]),
...
MS = ets:fun2ms(fun({N,_}=A) when (N rem 10) =:= 0 -> A end),
{_,C} = ets:select(T,MS,10),
MaybeBroken = binary_to_term(term_to_binary(C)),
ets:select(ets:repair_continuation(MaybeBroken,MS)).

Note

This function is rarely needed in application code. It is used by Mnesia to provide distributed select/3 and select/1 sequences. A normal application would either use Mnesia or keep the continuation from being converted to external format.

The actual behavior of compiled match specifications when recreated from external format has changed and may change in future releases, but this interface remains for backward compatibility. See is_compiled_ms/1.

safe_fixtable(Table, Fix)

-spec safe_fixtable(Table, Fix) -> true when Table :: table(), Fix :: boolean().

Fixes a table of type set, bag, or duplicate_bag for safe traversal using first/1 & next/2, match/3 & match/1, match_object/3 & match_object/1, or select/3 & select/1.

A process fixes a table by calling safe_fixtable(Table, true). The table remains fixed until the process releases it by calling safe_fixtable(Table, false), or until the process terminates.

If many processes fix a table, the table remains fixed until all processes have released it (or terminated). A reference counter is kept on a per process basis, and N consecutive fixes requires N releases to release the table.

When a table is fixed, a sequence of first/1 and next/2 calls are guaranteed to succeed even if keys are removed during the traversal. The keys for objects inserted or deleted during a traversal may or may not be returned by next/2 depending on the ordering of keys within the table and if the key exists at the time next/2 is called.

Example:

clean_all_with_value(Table,X) ->
    safe_fixtable(Table,true),
    clean_all_with_value(Table,X,ets:first(Table)),
    safe_fixtable(Table,false).

clean_all_with_value(Table,X,'$end_of_table') ->
    true;
clean_all_with_value(Table,X,Key) ->
    case ets:lookup(Table,Key) of
        [{Key,X}] ->
            ets:delete(Table,Key);
        _ ->
            true
    end,
    clean_all_with_value(Table,X,ets:next(Table,Key)).

Notice that deleted objects are not freed from a fixed table until it has been released. If a process fixes a table but never releases it, the memory used by the deleted objects is never freed. The performance of operations on the table also degrades significantly.

To retrieve information about which processes have fixed which tables, use info(Table, safe_fixed_monotonic_time). A system with many processes fixing tables can need a monitor that sends alarms when tables have been fixed for too long.

Notice that safe_fixtable/2 is not necessary for table type ordered_set and for traversals done by a single ETS function call, like select/2.

select(Continuation)

-spec select(Continuation) -> {[Match], Continuation} | '$end_of_table'
                when Match :: term(), Continuation :: continuation().

Continues a match started with select/3. The next chunk of the size specified in the initial select/3 call is returned together with a new Continuation, which can be used in subsequent calls to this function.

When there are no more objects in the table, '$end_of_table' is returned.

select(Table, MatchSpec)

-spec select(Table, MatchSpec) -> [Match]
                when Table :: table(), MatchSpec :: match_spec(), Match :: term().

Matches the objects in table Table using a match specification. This is a more general call than match/2 and match_object/2 calls. In its simplest form, the match specification is as follows:

MatchSpec = [MatchFunction]
MatchFunction = {MatchHead, [Guard], [Result]}
MatchHead = "Pattern as in ets:match"
Guard = {"Guardtest name", ...}
Result = "Term construct"

This means that the match specification is always a list of one or more tuples (of arity 3). The first element of the tuple is to be a pattern as described in match/2. The second element of the tuple is to be a list of 0 or more guard tests (described below). The third element of the tuple is to be a list containing a description of the value to return. In almost all normal cases, the list contains exactly one term that fully describes the value to return for each object.

The return value is constructed using the "match variables" bound in MatchHead or using the special match variables '$_' (the whole matching object) and '$$' (all match variables in a list), so that the following match/2 expression:

ets:match(Table,{'$1','$2','$3'})

is exactly equivalent to:

ets:select(Table,[{{'$1','$2','$3'},[],['$$']}])

And that the following match_object/2 call:

ets:match_object(Table,{'$1','$2','$1'})

is exactly equivalent to

ets:select(Table,[{{'$1','$2','$1'},[],['$_']}])

Composite terms can be constructed in the Result part either by simply writing a list, so that the following code:

ets:select(Table,[{{'$1','$2','$3'},[],['$$']}])

gives the same output as:

ets:select(Table,[{{'$1','$2','$3'},[],[['$1','$2','$3']]}])

That is, all the bound variables in the match head as a list. If tuples are to be constructed, one has to write a tuple of arity 1 where the single element in the tuple is the tuple one wants to construct (as an ordinary tuple can be mistaken for a Guard).

Therefore the following call:

ets:select(Table,[{{'$1','$2','$1'},[],['$_']}])

gives the same output as:

ets:select(Table,[{{'$1','$2','$1'},[],[{{'$1','$2','$3'}}]}])

This syntax is equivalent to the syntax used in the trace patterns (see the dbg) module in Runtime_Tools.

The Guards are constructed as tuples, where the first element is the test name and the remaining elements are the test parameters. To check for a specific type (say a list) of the element bound to the match variable '$1', one would write the test as {is_list, '$1'}. If the test fails, the object in the table does not match and the next MatchFunction (if any) is tried. Most guard tests present in Erlang can be used, but only the new versions prefixed is_ are allowed (is_float, is_atom, and so on).

The Guard section can also contain logic and arithmetic operations, which are written with the same syntax as the guard tests (prefix notation), so that the following guard test written in Erlang:

is_integer(X), is_integer(Y), X + Y < 4711

is expressed as follows (X replaced with '$1' and Y with '$2'):

[{is_integer, '$1'}, {is_integer, '$2'}, {'<', {'+', '$1', '$2'}, 4711}]

For tables of type ordered_set, objects are visited in the same order as in a first/next traversal. This means that the match specification is executed against objects with keys in the first/next order and the corresponding result list is in the order of that execution.

select(Table, MatchSpec, Limit)

-spec select(Table, MatchSpec, Limit) -> {[Match], Continuation} | '$end_of_table'
                when
                    Table :: table(),
                    MatchSpec :: match_spec(),
                    Limit :: pos_integer(),
                    Match :: term(),
                    Continuation :: continuation().

Works like select/2, but only returns a limited (Limit) number of matching objects. Term Continuation can then be used in subsequent calls to select/1 to get the next chunk of matching objects. This is a space-efficient way to work on objects in a table, which is still faster than traversing the table object by object using first/1 and next/2.

If the table is empty, '$end_of_table' is returned.

Use safe_fixtable/2 to guarantee safe traversal for subsequent calls to select/1.

select_count(Table, MatchSpec)

-spec select_count(Table, MatchSpec) -> NumMatched
                      when Table :: table(), MatchSpec :: match_spec(), NumMatched :: non_neg_integer().

Matches the objects in table Table using a match specification. If and only if the match specification returns true for an object, that object is considered a match and is counted. For any other result from the match specification the object is not considered a match and is therefore not counted.

The function returns the number of objects matched.

select_delete(Table, MatchSpec)

-spec select_delete(Table, MatchSpec) -> NumDeleted
                       when Table :: table(), MatchSpec :: match_spec(), NumDeleted :: non_neg_integer().

Matches the objects in table Table using a match specification. If and only if the match specification returns true for an object, that object is removed from the table. For any other result from the match specification the object is retained. This is a more general function than match_delete/2.

The function returns the number of objects deleted from the table.

Note

The match specification has to return the atom true if the object is to be deleted. No other return value gets the object deleted. So one cannot use the same match specification for looking up elements as for deleting them.

select_replace(Table, MatchSpec)

(since OTP 20.0)

-spec select_replace(Table, MatchSpec) -> NumReplaced
                        when
                            Table :: table(),
                            MatchSpec :: match_spec(),
                            NumReplaced :: non_neg_integer().

Matches the objects in the table Table using a match specification. For each matched object, the existing object is replaced with the match specification result.

The match-and-replace operation for each individual object is guaranteed to be atomic and isolated. The select_replace table traversal as a whole, like all other select functions, does not give such guarantees.

The match specification must be guaranteed to retain the key of any matched object. If not, select_replace will fail with badarg without updating any objects.

For the moment, due to performance and semantic constraints, tables of type bag are not yet supported.

The function returns the total number of replaced objects.

Example

For all 2-tuples with a list in second position, add atom 'marker' first in the list:

1> T = ets:new(x,[]), ets:insert(T, {key, [1, 2, 3]}).
true
2> MS = ets:fun2ms(fun({K, L}) when is_list(L) -> {K, [marker | L]} end).
[{{'$1','$2'},[{is_list,'$2'}],[{{'$1',[marker|'$2']}}]}]
3> ets:select_replace(T, MS).
1
4> ets:tab2list(T).
[{key,[marker,1,2,3]}]

A generic single object compare-and-swap operation:

[Old] = ets:lookup(T, Key),
New = update_object(Old),
Success = (1 =:= ets:select_replace(T, [{Old, [], [{const, New}]}])),

select_reverse(Continuation)

(since OTP R14B)

-spec select_reverse(Continuation) -> {[Match], Continuation} | '$end_of_table'
                        when Continuation :: continuation(), Match :: term().

Continues a match started with select_reverse/3. For tables of type ordered_set, the traversal of the table continues to objects with keys earlier in the Erlang term order. The returned list also contains objects with keys in reverse order. For all other table types, the behavior is exactly that of select/1.

Example:

1> T = ets:new(x,[ordered_set]).
2> [ ets:insert(T,{N}) || N <- lists:seq(1,10) ].
...
3> {R0,C0} = ets:select_reverse(T,[{'_',[],['$_']}],4).
...
4> R0.
[{10},{9},{8},{7}]
5> {R1,C1} = ets:select_reverse(C0).
...
6> R1.
[{6},{5},{4},{3}]
7> {R2,C2} = ets:select_reverse(C1).
...
8> R2.
[{2},{1}]
9> '$end_of_table' = ets:select_reverse(C2).
...

select_reverse(Table, MatchSpec)

(since OTP R14B)

-spec select_reverse(Table, MatchSpec) -> [Match]
                        when Table :: table(), MatchSpec :: match_spec(), Match :: term().

Works like select/2, but returns the list in reverse order for table type ordered_set. For all other table types, the return value is identical to that of select/2.

select_reverse(Table, MatchSpec, Limit)

(since OTP R14B)

-spec select_reverse(Table, MatchSpec, Limit) -> {[Match], Continuation} | '$end_of_table'
                        when
                            Table :: table(),
                            MatchSpec :: match_spec(),
                            Limit :: pos_integer(),
                            Match :: term(),
                            Continuation :: continuation().

Works like select/3, but for table type ordered_set traversing is done starting at the last object in Erlang term order and moves to the first. For all other table types, the return value is identical to that of select/3.

Notice that this is not equivalent to reversing the result list of a select/3 call, as the result list is not only reversed, but also contains the last Limit matching objects in the table, not the first.

setopts(Table, Opts)

-spec setopts(Table, Opts) -> true
                 when
                     Table :: table(),
                     Opts :: Opt | [Opt],
                     Opt :: {heir, Pid} | {heir, Pid, HeirData} | {heir, none},
                     Pid :: pid(),
                     HeirData :: term().

Sets table options. The only allowed option to be set after the table has been created is heir. The calling process must be the table owner.

slot(Table, I)

-spec slot(Table, I) -> [Object] | '$end_of_table'
              when Table :: table(), I :: non_neg_integer(), Object :: tuple().

This function is mostly for debugging purposes, normally first/next or last/prev are to be used instead.

Returns all objects in slot I of table Table. A table can be traversed by repeatedly calling the function, starting with the first slot I=0 and ending when '$end_of_table' is returned. If argument I is out of range, the function fails with reason badarg.

Unless a table of type set, bag, or duplicate_bag is protected using safe_fixtable/2, a traversal can fail if concurrent updates are made to the table. For table type ordered_set, the function returns a list containing object I in Erlang term order.

tab2file(Table, Filename)

-spec tab2file(Table, Filename) -> ok | {error, Reason}
                  when Table :: table(), Filename :: file:name(), Reason :: term().

Dumps table Table to file Filename.

Equivalent to tab2file(Table, Filename,[])

tab2file(Table, Filename, Options)

-spec tab2file(Table, Filename, Options) -> ok | {error, Reason}
                  when
                      Table :: table(),
                      Filename :: file:name(),
                      Options :: [Option],
                      Option :: {extended_info, [ExtInfo]} | {sync, boolean()},
                      ExtInfo :: md5sum | object_count,
                      Reason :: term().

Dumps table Table to file Filename.

When dumping the table, some information about the table is dumped to a header at the beginning of the dump. This information contains data about the table type, name, protection, size, version, and if it is a named table. It also contains notes about what extended information is added to the file, which can be a count of the objects in the file or a MD5 sum of the header and records in the file.

The size field in the header might not correspond to the number of records in the file if the table is public and records are added or removed from the table during dumping. Public tables updated during dump, and that one wants to verify when reading, needs at least one field of extended information for the read verification process to be reliable later.

Option extended_info specifies what extra information is written to the table dump:

object_count - The number of objects written to the file is noted in the file footer, so file truncation can be verified even if the file was updated during dump.

md5sum - The header and objects in the file are checksummed using the built-in MD5 functions. The MD5 sum of all objects is written in the file footer, so that verification while reading detects the slightest bitflip in the file data. Using this costs a fair amount of CPU time.

Whenever option extended_info is used, it results in a file not readable by versions of ETS before that in STDLIB 1.15.1

If option sync is set to true, it ensures that the content of the file is written to the disk before tab2file returns. Defaults to {sync, false}.

tab2list(Table)

-spec tab2list(Table) -> [Object] when Table :: table(), Object :: tuple().

Returns a list of all objects in table Table.

tabfile_info(Filename)

-spec tabfile_info(Filename) -> {ok, TableInfo} | {error, Reason}
                      when
                          Filename :: file:name(),
                          TableInfo :: [InfoItem],
                          InfoItem ::
                              {name, atom()} |
                              {type, Type} |
                              {protection, Protection} |
                              {named_table, boolean()} |
                              {keypos, non_neg_integer()} |
                              {size, non_neg_integer()} |
                              {extended_info, [ExtInfo]} |
                              {version, {Major :: non_neg_integer(), Minor :: non_neg_integer()}},
                          ExtInfo :: md5sum | object_count,
                          Type :: bag | duplicate_bag | ordered_set | set,
                          Protection :: private | protected | public,
                          Reason :: term().

Returns information about the table dumped to file by tab2file/2 or tab2file/3.

The following items are returned:

name - The name of the dumped table. If the table was a named table, a table with the same name cannot exist when the table is loaded from file with file2tab/2. If the table is not saved as a named table, this field has no significance when loading the table from file.

type - The ETS type of the dumped table (that is, set, bag, duplicate_bag, or ordered_set). This type is used when loading the table again.

protection - The protection of the dumped table (that is, private, protected, or public). A table loaded from the file gets the same protection.

named_table - true if the table was a named table when dumped to file, otherwise false. Notice that when a named table is loaded from a file, there cannot exist a table in the system with the same name.

keypos - The keypos of the table dumped to file, which is used when loading the table again.

size - The number of objects in the table when the table dump to file started. For a public table, this number does not need to correspond to the number of objects saved to the file, as objects can have been added or deleted by another process during table dump.

extended_info - The extended information written in the file footer to allow stronger verification during table loading from file, as specified to tab2file/3. Notice that this function only tells which information is present, not the values in the file footer. The value is a list containing one or more of the atoms object_count and md5sum.

version - A tuple {Major,Minor} containing the major and minor version of the file format for ETS table dumps. This version field was added beginning with STDLIB 1.5.1. Files dumped with older versions return {0,0} in this field.

An error is returned if the file is inaccessible, badly damaged, or not produced with tab2file/2 or tab2file/3.

table(Table)

-spec table(Table) -> QueryHandle when Table :: table(), QueryHandle :: qlc:query_handle().

Equivalent to table/2.

table(Table, Options)

-spec table(Table, Options) -> QueryHandle
               when
                   Table :: table(),
                   QueryHandle :: qlc:query_handle(),
                   Options :: [Option] | Option,
                   Option :: {n_objects, NObjects} | {traverse, TraverseMethod},
                   NObjects :: default | pos_integer(),
                   TraverseMethod ::
                       first_next | last_prev | select | {select, MatchSpec :: match_spec()}.

Returns a Query List Comprehension (QLC) query handle. The qlc module provides a query language aimed mainly at Mnesia, but ETS tables, Dets tables, and lists are also recognized by QLC as sources of data. Calling table/1,2 is the means to make the ETS table Table usable to QLC.

When there are only simple restrictions on the key position, QLC uses lookup/2 to look up the keys. When that is not possible, the whole table is traversed. Option traverse determines how this is done:

first_next - The table is traversed one key at a time by calling first/1 and next/2.

last_prev - The table is traversed one key at a time by calling last/1 and prev/2.

select - The table is traversed by calling select/3 and select/1. Option n_objects determines the number of objects returned (the third argument of select/3); the default is to return 100 objects at a time. The match specification (the second argument of select/3) is assembled by QLC: simple filters are translated into equivalent match specifications while more complicated filters must be applied to all objects returned by select/3 given a match specification that matches all objects.

{select, MatchSpec} - As for select, the table is traversed by calling select/3 and select/1. The difference is that the match specification is explicitly specified. This is how to state match specifications that cannot easily be expressed within the syntax provided by QLC.

Examples:

An explicit match specification is here used to traverse the table:

9> true = ets:insert(Table = ets:new(t, []), [{1,a},{2,b},{3,c},{4,d}]),
MS = ets:fun2ms(fun({X,Y}) when (X > 1) or (X < 5) -> {Y} end),
QH1 = ets:table(Table, [{traverse, {select, MS}}]).

An example with an implicit match specification:

10> QH2 = qlc:q([{Y} || {X,Y} <- ets:table(Table), (X > 1) or (X < 5)]).

The latter example is equivalent to the former, which can be verified using function qlc:info/1:

11> qlc:info(QH1) =:= qlc:info(QH2).
true

qlc:info/1 returns information about a query handle, and in this case identical information is returned for the two query handles.

take(Table, Key)

(since OTP 18.0)

-spec take(Table, Key) -> [Object] when Table :: table(), Key :: term(), Object :: tuple().

Returns and removes a list of all objects with key Key in table Table.

The specified Key is used to identify the object by either comparing equal the key of an object in an ordered_set table, or matching in other types of tables (for details on the difference, see lookup/2 and new/2).

test_ms(Tuple, MatchSpec)

-spec test_ms(Tuple, MatchSpec) -> {ok, Result} | {error, Errors}
                 when
                     Tuple :: tuple(),
                     MatchSpec :: match_spec(),
                     Result :: term(),
                     Errors :: [{warning | error, string()}].

This function is a utility to test a match specification used in calls to select/2. The function both tests MatchSpec for "syntactic" correctness and runs the match specification against object Tuple.

If the match specification is syntactically correct, the function either returns {ok,Result}, where Result is what would have been the result in a real select/2 call, or false if the match specification does not match object Tuple.

If the match specification contains errors, tuple {error, Errors} is returned, where Errors is a list of natural language descriptions of what was wrong with the match specification.

This is a useful debugging and test tool, especially when writing complicated select/2 calls.

See also: erlang:match_spec_test/3.

to_dets(Table, DetsTab)

-spec to_dets(Table, DetsTab) -> DetsTab when Table :: table(), DetsTab :: dets:tab_name().

Fills an already created/opened Dets table with the objects in the already opened ETS table named Table. The Dets table is emptied before the objects are inserted.

update_counter/3

-spec update_counter(Table, Key, UpdateOp | [UpdateOp] | Incr) -> Result | [Result]
                        when
                            Table :: table(),
                            Key :: term(),
                            UpdateOp :: {Pos, Incr} | {Pos, Incr, Threshold, SetValue},
                            Pos :: integer(),
                            Incr :: integer(),
                            Threshold :: integer(),
                            SetValue :: integer(),
                            Result :: integer().

Equivalent to update_counter/4.

update_counter/4

(since OTP 18.0)

-spec update_counter(Table, Key, UpdateOp | Incr | [UpdateOp], Default) -> Result | [Result]
                        when
                            Table :: table(),
                            Key :: term(),
                            UpdateOp :: {Pos, Incr} | {Pos, Incr, Threshold, SetValue},
                            Pos :: integer(),
                            Incr :: integer(),
                            Threshold :: integer(),
                            SetValue :: integer(),
                            Result :: integer(),
                            Default :: tuple().

This function provides an efficient way to update one or more counters, without the trouble of having to look up an object, update the object by incrementing an element, and insert the resulting object into the table again. The operation is guaranteed to be atomic and isolated.

This function destructively updates the object with key Key in table Table by adding Incr to the element at position Pos. The new counter value is returned. If no position is specified, the element directly following key (<keypos>+1) is updated.

If a Threshold is specified, the counter is reset to value SetValue if the following conditions occur:

Incr is not negative (>= 0) and the result would be greater than (>) Threshold.

Incr is negative (< 0) and the result would be less than (<) Threshold.

A list of UpdateOp can be supplied to do many update operations within the object. The operations are carried out in the order specified in the list. If the same counter position occurs more than once in the list, the corresponding counter is thus updated many times, each time based on the previous result. The return value is a list of the new counter values from each update operation in the same order as in the operation list. If an empty list is specified, nothing is updated and an empty list is returned. If the function fails, no updates are done.

The specified Key is used to identify the object by either matching the key of an object in a set table, or compare equal to the key of an object in an ordered_set table (for details on the difference, see lookup/2 and new/2).

If a default object Default is specified, it is used as the object to be updated if the key is missing from the table. The value in place of the key is ignored and replaced by the proper key value. The return value is as if the default object had not been used, that is, a single updated element or a list of them.

The function fails with reason badarg in the following situations:

The table type is not set or ordered_set.

No object with the correct key exists and no default object was supplied.

The object has the wrong arity.

The default object arity is smaller than <keypos>.

Any field from the default object that is updated is not an integer.

The element to update is not an integer.

The element to update is also the key.

Any of Pos, Incr, Threshold, or SetValue is not an integer.

update_element(Table, Key, ElementSpec)

-spec update_element(Table, Key, ElementSpec) -> boolean()
                        when
                            Table :: table(),
                            Key :: term(),
                            ElementSpec :: {Pos, Value} | [{Pos, Value}],
                            Pos :: pos_integer(),
                            Value :: term().

Equivalent to update_element/4.

update_element(Table, Key, ElementSpec, Default)

(since OTP 27.0)

-spec update_element(Table, Key, ElementSpec, Default) -> boolean()
                        when
                            Table :: table(),
                            Key :: term(),
                            ElementSpec :: {Pos, Value} | [{Pos, Value}],
                            Pos :: pos_integer(),
                            Value :: term(),
                            Default :: tuple().

This function provides an efficient way to update one or more elements within an object, without the trouble of having to look up, update, and write back the entire object.

This function destructively updates the object with key Key in table Table. The element at position Pos is given the value Value.

A list of {Pos,Value} can be supplied to update many elements within the same object. If the same position occurs more than once in the list, the last value in the list is written. If the list is empty or the function fails, no updates are done. The function is also atomic in the sense that other processes can never see any intermediate results.

Returns true if an object with key Key is found, otherwise false.

The specified Key is used to identify the object by either matching the key of an object in a set table, or compare equal to the key of an object in an ordered_set table (for details on the difference, see lookup/2 and new/2).

If a default object Default is specified, it is used as the object to be updated if the key is missing from the table. The value in place of the key is ignored and replaced by the proper key value.

The function fails with reason badarg in the following situations:

The table type is not set or ordered_set.

Pos < 1.

Pos > object arity.

The default object arity is smaller than <keypos>.

The element to update is also the key.

whereis(TableName)

(since OTP 21.0)

-spec whereis(TableName) -> tid() | undefined when TableName :: atom().

This function returns the tid/0 of the named table identified by TableName, or undefined if no such table exists. The tid/0 can be used in place of the table name in all operations, which is slightly faster since the name does not have to be resolved on each call.

If the table is deleted, the tid/0 will be invalid even if another named table is created with the same name.

gb_sets

Sets represented by general balanced trees.

This module provides ordered sets using Prof. Arne Andersson's General Balanced Trees. Ordered sets can be much more efficient than using ordered lists, for larger sets, but depends on the application.

The data representing a set as used by this module is to be regarded as opaque by other modules. In abstract terms, the representation is a composite type of existing Erlang terms. See note on data types. Any code assuming knowledge of the format is running on thin ice.

This module considers two elements as different if and only if they do not compare equal (==).

Complexity Note

The complexity on set operations is bounded by either O(|S|) or O(|T| log(|S|))*, where S is the largest given set, depending on which is fastest for any particular function call. For operating on sets of almost equal size, this implementation is about 3 times slower than using ordered-list sets directly. For sets of very different sizes, however, this solution can be arbitrarily much faster; in practical cases, often 10-100 times. This implementation is particularly suited for accumulating elements a few at a time, building up a large set (> 100-200 elements), and repeatedly testing for membership in the current set.

As with normal tree structures, lookup (membership testing), insertion, and deletion have logarithmic complexity.

Compatibility

See the Compatibility Section in the sets module for information about the compatibility of the different implementations of sets in the Standard Library.

See Also

gb_trees, ordsets, sets

Summary

Types

iter()

iter(Element)

A general balanced set iterator.

set()

set(Element)

A general balanced set.

Functions

add(Element, Set1)

Equivalent to add_element(Element, Set1).

add_element(Element, Set1)

Returns a new set formed from Set1 with Element inserted.

balance(Set1)

Rebalances the tree representation of Set1.

del_element(Element, Set1)

Equivalent to delete_any(Element, Set1).

delete(Element, Set1)

Returns a new set formed from Set1 with Element removed, assuming Element is present in Set1.

delete_any(Element, Set1)

Returns a new set formed from Set1 with Element removed.

difference(Set1, Set2)

Equivalent to subtract(Set1, Set2).

empty()

Returns a new empty set.

filter(Pred, Set1)

Filters elements in Set1 using predicate function Pred.

filtermap(Fun, Set1)

Calls Fun(Elem) for each Elem of Set1 to update or remove elements from Set1.

fold(Function, Acc0, Set)

Folds Function over every element in Set and returns the final value of the accumulator.

from_list(List)

Returns a set of the elements in List, where List can be unordered and contain duplicates.

from_ordset(List)

Turns an ordered list without duplicates List into a set.

insert(Element, Set1)

Returns a new set formed from Set1 with Element inserted, assuming Element is not already present.

intersection(SetList)

Returns the intersection of the non-empty list of sets.

intersection(Set1, Set2)

Returns the intersection of Set1 and Set2.

is_disjoint(Set1, Set2)

Returns true if Set1 and Set2 are disjoint; otherwise, returns false.

is_element(Element, Set)

Equivalent to is_member(Element, Set).

is_empty(Set)

Returns true if Set is an empty set; otherwise, returns false.

is_equal(Set1, Set2)

Returns true if Set1 and Set2 are equal, that is, if every element of one set is also a member of the other set; otherwise, returns false.

is_member(Element, Set)

Returns true if Element is an element of Set; otherwise, returns false.

is_set(Term)

Returns true if Term appears to be a set; otherwise, returns false.

is_subset(Set1, Set2)

Returns true when every element of Set1 is also a member of Set2; otherwise, returns false.

iterator(Set)

Returns an iterator that can be used for traversing the entries of Set; see next/1.

iterator(Set, Order)

Returns an iterator that can be used for traversing the entries of Set in either ordered or reversed direction; see next/1.

iterator_from(Element, Set)

Returns an iterator that can be used for traversing the entries of Set; see next/1.

iterator_from(Element, Set, Order)

Returns an iterator that can be used for traversing the entries of Set; see next/1.

larger(Element1, Set)

Returns {found, Element2}, where Element2 is the least element strictly greater than Element1.

largest(Set)

Returns the largest element in Set.

map(Fun, Set1)

Maps elements in Set1 with mapping function Fun.

new()

Returns a new empty set.

next(Iter1)

Returns {Element, Iter2}, where Element is the first element referred to by iterator Iter1, and Iter2 is the new iterator to be used for traversing the remaining elements, or the atom none if no elements remain.

singleton(Element)

Returns a set containing only element Element.

size(Set)

Returns the number of elements in Set.

smaller(Element1, Set)

Returns {found, Element2}, where Element2 is the greatest element strictly less than Element1.

smallest(Set)

Returns the smallest element in Set.

subtract(Set1, Set2)

Returns a new set containing the elements of Set1 that are not elements in Set2.

take_largest(Set1)

Returns {Element, Set2}, where Element is the largest element in Set1, and Set2 is this set with Element deleted.

take_smallest(Set1)

Returns {Element, Set2}, where Element is the smallest element in Set1, and Set2 is this set with Element deleted.

to_list(Set)

Returns the elements of Set as an ordered list.

union(SetList)

Returns the union of a list of sets.

union(Set1, Set2)

Returns the union of Set1 and Set2.

Types

iter()

-type iter() :: iter(_).

iter(Element)

-opaque iter(Element)

A general balanced set iterator.

set()

-type set() :: set(_).

set(Element)

-opaque set(Element)

A general balanced set.

Functions

add(Element, Set1)

-spec add(Element, Set1) -> Set2 when Set1 :: set(Element), Set2 :: set(Element).

Equivalent to add_element(Element, Set1).

add_element(Element, Set1)

-spec add_element(Element, Set1) -> Set2 when Set1 :: set(Element), Set2 :: set(Element).

Returns a new set formed from Set1 with Element inserted.

If Element is already an element in Set1, nothing is changed.

Examples

1> S0 = gb_sets:new().
2> S1 = gb_sets:add_element(7, S0).
3> gb_sets:to_list(S1).
[7]
4> S2 = gb_sets:add_element(42, S1).
5> S2 = gb_sets:add_element(42, S1).
6> gb_sets:to_list(S2).
[7,42]

balance(Set1)

-spec balance(Set1) -> Set2 when Set1 :: set(Element), Set2 :: set(Element).

Rebalances the tree representation of Set1.

This is rarely necessary, but can be motivated when a large number of elements have been deleted from the tree without further insertions. Forcing rebalancing can minimize lookup times, as deletion does not rebalance the tree.

Examples

1> S0 = gb_sets:from_ordset(lists:seq(1, 100)).
2> Delete = fun(E, Set) -> gb_sets:delete(E, Set) end.
3> S1 = lists:foldl(Delete, S0, lists:seq(1, 50)).
4> gb_sets:size(S1).
50
5> S2 = gb_sets:balance(S1).

del_element(Element, Set1)

-spec del_element(Element, Set1) -> Set2 when Set1 :: set(Element), Set2 :: set(Element).

Equivalent to delete_any(Element, Set1).

delete(Element, Set1)

-spec delete(Element, Set1) -> Set2 when Set1 :: set(Element), Set2 :: set(Element).

Returns a new set formed from Set1 with Element removed, assuming Element is present in Set1.

Use delete_any/2 when deleting from a set where Element is potentially missing.

Examples

1> S = gb_sets:from_list([a,b]).
2> gb_sets:to_list(gb_sets:delete(b, S)).
[a]

delete_any(Element, Set1)

-spec delete_any(Element, Set1) -> Set2 when Set1 :: set(Element), Set2 :: set(Element).

Returns a new set formed from Set1 with Element removed.

If Element is not an element in Set1, nothing is changed.

Examples

1> S = gb_sets:from_list([a,b]).
2> gb_sets:to_list(gb_sets:delete_any(b, S)).
[a]
3> S = gb_sets:delete_any(x, S).

difference(Set1, Set2)

-spec difference(Set1, Set2) -> Set3
                    when Set1 :: set(Element), Set2 :: set(Element), Set3 :: set(Element).

Equivalent to subtract(Set1, Set2).

empty()

-spec empty() -> Set when Set :: set(none()).

Returns a new empty set.

Examples

1> gb_sets:to_list(gb_sets:empty()).
[]

filter(Pred, Set1)

-spec filter(Pred, Set1) -> Set2
                when Pred :: fun((Element) -> boolean()), Set1 :: set(Element), Set2 :: set(Element).

Filters elements in Set1 using predicate function Pred.

Examples

1> S = gb_sets:from_list([1,2,3,4,5,6,7]).
2> IsEven = fun(N) -> N rem 2 =:= 0 end.
3> Filtered = gb_sets:filter(IsEven, S).
4> gb_sets:to_list(Filtered).
[2,4,6]

filtermap(Fun, Set1)

(since OTP 27.0)

-spec filtermap(Fun, Set1) -> Set2
                   when
                       Fun :: fun((Element1) -> boolean() | {true, Element2}),
                       Set1 :: set(Element1),
                       Set2 :: set(Element1 | Element2).

Calls Fun(Elem) for each Elem of Set1 to update or remove elements from Set1.

Fun/1 must return either a Boolean or a tuple {true, Value}. The function returns the set of elements for which Fun returns a new value, with true being equivalent to {true, Elem}.

gb_sets:filtermap/2 behaves as if it were defined as follows:

filtermap(Fun, Set1) ->
    gb_sets:from_list(lists:filtermap(Fun, Set1)).

Examples

1> S = gb_sets:from_list([2,4,5,6,8,9])
2> F = fun(X) ->
           case X rem 2 of
               0 -> {true, X div 2};
               1 -> false
           end
        end.
3> Set = gb_sets:filtermap(F, S).
4> gb_sets:to_list(Set).
[1,2,3,4]

fold(Function, Acc0, Set)

-spec fold(Function, Acc0, Set) -> Acc1
              when
                  Function :: fun((Element, AccIn) -> AccOut),
                  Acc0 :: Acc,
                  Acc1 :: Acc,
                  AccIn :: Acc,
                  AccOut :: Acc,
                  Set :: set(Element).

Folds Function over every element in Set and returns the final value of the accumulator.

Examples

1> S = gb_sets:from_list([1,2,3,4]).
2> Plus = fun erlang:'+'/2.
3> gb_sets:fold(Plus, 0, S).
10

from_list(List)

-spec from_list(List) -> Set when List :: [Element], Set :: set(Element).

Returns a set of the elements in List, where List can be unordered and contain duplicates.

Examples

1> Unordered = [x,y,a,x,y,b,b,z]
2> gb_sets:to_list(gb_sets:from_list(Unordered)).
[a,b,x,y,z]

from_ordset(List)

-spec from_ordset(List) -> Set when List :: [Element], Set :: set(Element).

Turns an ordered list without duplicates List into a set.

See from_list/1 for a function that accepts unordered lists with duplicates.

Examples

1> Ordset = [1,2,3].
2> gb_sets:to_list(gb_sets:from_ordset(Ordset)).
[1,2,3]

insert(Element, Set1)

-spec insert(Element, Set1) -> Set2 when Set1 :: set(Element), Set2 :: set(Element).

Returns a new set formed from Set1 with Element inserted, assuming Element is not already present.

Use add/2 for inserting into a set where Element is potentially already present.

Examples

1> S0 = gb_sets:new().
2> S1 = gb_sets:insert(7, S0).
3> gb_sets:to_list(S1).
[7]
4> S2 = gb_sets:insert(42, S1).
5> gb_sets:to_list(S2).
[7,42]

intersection(SetList)

-spec intersection(SetList) -> Set when SetList :: [set(Element), ...], Set :: set(Element).

Returns the intersection of the non-empty list of sets.

The intersection of multiple sets is a new set that contains only the elements that are present in all sets.

Examples

1> S0 = gb_sets:from_list([a,b,c,d]).
2> S1 = gb_sets:from_list([d,e,f]).
3> S2 = gb_sets:from_list([q,r])
4> Sets = [S0, S1, S2].
5> gb_sets:to_list(gb_sets:intersection([S0, S1, S2])).
[]
6> gb_sets:to_list(gb_sets:intersection([S0, S1])).
[d]
7> gb_sets:intersection([]).
** exception error: no function clause matching gb_sets:intersection([])

intersection(Set1, Set2)

-spec intersection(Set1, Set2) -> Set3
                      when Set1 :: set(Element), Set2 :: set(Element), Set3 :: set(Element).

Returns the intersection of Set1 and Set2.

The intersection of two sets is a new set that contains only the elements that are present in both sets.

Examples

1> S0 = gb_sets:from_list([a,b,c,d]).
2> S1 = gb_sets:from_list([c,d,e,f]).
3> S2 = gb_sets:from_list([q,r]).
4> gb_sets:to_list(gb_sets:intersection(S0, S1)).
[c,d]
5> gb_sets:to_list(gb_sets:intersection(S1, S2)).
[]

is_disjoint(Set1, Set2)

-spec is_disjoint(Set1, Set2) -> boolean() when Set1 :: set(Element), Set2 :: set(Element).

Returns true if Set1 and Set2 are disjoint; otherwise, returns false.

Two sets are disjoint if they have no elements in common.

This function is equivalent to gb_sets:intersection(Set1, Set2) =:= [], but faster.

Examples

1> S0 = gb_sets:from_list([a,b,c,d]).
2> S1 = gb_sets:from_list([d,e,f]).
3> S2 = gb_sets:from_list([q,r])
4> gb_sets:is_disjoint(S0, S1).
false
5> gb_sets:is_disjoint(S1, S2).
true

is_element(Element, Set)

-spec is_element(Element, Set) -> boolean() when Set :: set(Element).

Equivalent to is_member(Element, Set).

is_empty(Set)

-spec is_empty(Set) -> boolean() when Set :: set().

Returns true if Set is an empty set; otherwise, returns false.

Examples

1> gb_sets:is_empty(gb_sets:new()).
true
2> gb_sets:is_empty(gb_sets:singleton(1)).
false

is_equal(Set1, Set2)

(since OTP 27.0)

-spec is_equal(Set1, Set2) -> boolean() when Set1 :: set(), Set2 :: set().

Returns true if Set1 and Set2 are equal, that is, if every element of one set is also a member of the other set; otherwise, returns false.

Examples

1> Empty = gb_sets:new().
2> S = gb_sets:from_list([a,b]).
3> gb_sets:is_equal(S, S)
true
4> gb_sets:is_equal(S, Empty)
false

is_member(Element, Set)

-spec is_member(Element, Set) -> boolean() when Set :: set(Element).

Returns true if Element is an element of Set; otherwise, returns false.

Examples

1> S = gb_sets:from_list([a,b,c]).
2> gb_sets:is_member(42, S).
false
3> gb_sets:is_member(b, S).
true

is_set(Term)

-spec is_set(Term) -> boolean() when Term :: term().

Returns true if Term appears to be a set; otherwise, returns false.

Note

This function will return true for any term that coincides with the representation of a gb_set, while not really being a gb_set, thus it might return false positive results. See also note on data types.

Furthermore, since gb_sets are opaque, calling this function on terms that are not gb_sets could result in dialyzer warnings.

Examples

1> gb_sets:is_set(gb_sets:new()).
true
2> gb_sets:is_set(gb_sets:singleton(42)).
true
3> gb_sets:is_set(0).
false

is_subset(Set1, Set2)

-spec is_subset(Set1, Set2) -> boolean() when Set1 :: set(Element), Set2 :: set(Element).

Returns true when every element of Set1 is also a member of Set2; otherwise, returns false.

Examples

1> S0 = gb_sets:from_list([a,b,c,d]).
2> S1 = gb_sets:from_list([c,d]).
3> gb_sets:is_subset(S1, S0).
true
4> gb_sets:is_subset(S0, S1).
false
5> gb_sets:is_subset(S0, S0).
true

iterator(Set)

-spec iterator(Set) -> Iter when Set :: set(Element), Iter :: iter(Element).

Returns an iterator that can be used for traversing the entries of Set; see next/1.

Equivalent to iterator(Set, ordered).

iterator(Set, Order)

(since OTP 27.0)

-spec iterator(Set, Order) -> Iter
                  when Set :: set(Element), Iter :: iter(Element), Order :: ordered | reversed.

Returns an iterator that can be used for traversing the entries of Set in either ordered or reversed direction; see next/1.

The implementation is very efficient; traversing the whole set using next/1 is only slightly slower than getting the list of all elements using to_list/1 and traversing that. The main advantage of the iterator approach is that it avoids building the complete list of all elements to be built in memory at once.

1> S = gb_sets:from_ordset([1,2,3,4,5]).
2> Iter0 = gb_sets:iterator(S, ordered).
3> element(1, gb_sets:next(Iter0)).
1
4> Iter1 = gb_sets:iterator(S, reversed).
5> element(1, gb_sets:next(Iter1)).
5

iterator_from(Element, Set)

(since OTP 18.0)

-spec iterator_from(Element, Set) -> Iter when Set :: set(Element), Iter :: iter(Element).

Returns an iterator that can be used for traversing the entries of Set; see next/1.

Unlike the iterator returned by iterator/1 or iterator/2, this iterator starts with the first element greater than or equal to Element.

Equivalent to iterator_from(Element, Set, ordered).

Examples

1> S = gb_sets:from_ordset([10,20,30,40,50]).
2> Iter = gb_sets:iterator_from(17, S).
3> element(1, gb_sets:next(Iter)).
20

iterator_from(Element, Set, Order)

(since OTP 27.0)

-spec iterator_from(Element, Set, Order) -> Iter
                       when Set :: set(Element), Iter :: iter(Element), Order :: ordered | reversed.

Returns an iterator that can be used for traversing the entries of Set; see next/1.

Unlike the iterator returned by iterator/1 or iterator/2, this iterator starts with the first element greater than or equal to Element.

Examples

1> S = gb_sets:from_ordset([10,20,30,40,50]).
2> Iter = gb_sets:iterator_from(17, S, reversed).
3> element(1, gb_sets:next(Iter)).
10

larger(Element1, Set)

(since OTP 27.0)

-spec larger(Element1, Set) -> none | {found, Element2}
                when Element1 :: Element, Element2 :: Element, Set :: set(Element).

Returns {found, Element2}, where Element2 is the least element strictly greater than Element1.

Returns none if no such element exists.

Examples

1> S = gb_sets:from_list([10,20,30]).
2> gb_sets:larger(1, S).
{found,10}
3> gb_sets:larger(10, S).
{found,20}
4> gb_sets:larger(19, S).
{found,20}
5> gb_sets:larger(30, S).
none

largest(Set)

-spec largest(Set) -> Element when Set :: set(Element).

Returns the largest element in Set.

Assumes that Set is not empty.

Examples

1> S = gb_sets:from_list([a,b,c]).
2> gb_sets:largest(S).
c

map(Fun, Set1)

(since OTP 27.0)

-spec map(Fun, Set1) -> Set2
             when Fun :: fun((Element1) -> Element2), Set1 :: set(Element1), Set2 :: set(Element2).

Maps elements in Set1 with mapping function Fun.

Examples

1> S = gb_sets:from_list([1,2,3,4,5,6,7]).
2> F = fun(N) -> N div 2 end.
3> Mapped = gb_sets:map(F, S).
4> gb_sets:to_list(Mapped).
[0,1,2,3]

new()

-spec new() -> Set when Set :: set(none()).

Returns a new empty set.

Examples

1> gb_sets:to_list(gb_sets:new()).
[]

next(Iter1)

-spec next(Iter1) -> {Element, Iter2} | none when Iter1 :: iter(Element), Iter2 :: iter(Element).

Returns {Element, Iter2}, where Element is the first element referred to by iterator Iter1, and Iter2 is the new iterator to be used for traversing the remaining elements, or the atom none if no elements remain.

1> S = gb_sets:from_ordset([1,2,3,4,5]).
2> Iter0 = gb_sets:iterator(S).
3> {Element0, Iter1} = gb_sets:next(Iter0).
4> Element0.
1
5> {Element1, Iter2} = gb_sets:next(Iter1).
6> Element1.
2

singleton(Element)

-spec singleton(Element) -> set(Element).

Returns a set containing only element Element.

Examples

1> S = gb_sets:singleton(42).
2> gb_sets:to_list(S).
[42]

size(Set)

-spec size(Set) -> non_neg_integer() when Set :: set().

Returns the number of elements in Set.

Examples

1> gb_sets:size(gb_sets:new()).
0
2> gb_sets:size(gb_sets:from_list([4,5,6])).
3

smaller(Element1, Set)

(since OTP 27.0)

-spec smaller(Element1, Set) -> none | {found, Element2}
                 when Element1 :: Element, Element2 :: Element, Set :: set(Element).

Returns {found, Element2}, where Element2 is the greatest element strictly less than Element1.

Returns none if no such element exists.

Examples

1> S = gb_sets:from_list([a,b,c]).
2> gb_sets:smaller(b, S).
{found,a}
3> gb_sets:smaller(z, S).
{found,c}
4> gb_sets:smaller(a, S).
none

smallest(Set)

-spec smallest(Set) -> Element when Set :: set(Element).

Returns the smallest element in Set.

Assumes that Set is not empty.

Examples

1> S = gb_sets:from_list([a,b,c]).
2> gb_sets:smallest(S).
a

subtract(Set1, Set2)

-spec subtract(Set1, Set2) -> Set3 when Set1 :: set(Element), Set2 :: set(Element), Set3 :: set(Element).

Returns a new set containing the elements of Set1 that are not elements in Set2.

Examples

1> S0 = gb_sets:from_list([a,b,c,d]).
2> S1 = gb_sets:from_list([c,d,e,f]).
3> gb_sets:to_list(gb_sets:subtract(S0, S1)).
[a,b]
4> gb_sets:to_list(gb_sets:subtract(S1, S0)).
[e,f]

take_largest(Set1)

-spec take_largest(Set1) -> {Element, Set2} when Set1 :: set(Element), Set2 :: set(Element).

Returns {Element, Set2}, where Element is the largest element in Set1, and Set2 is this set with Element deleted.

Assumes that Set1 is not empty.

Examples

1> S0 = gb_sets:from_list([a,b,c]).
2> {Largest,S1} = gb_sets:take_largest(S0).
3> Largest.
c
4> gb_sets:to_list(S1).
[a,b]

take_smallest(Set1)

-spec take_smallest(Set1) -> {Element, Set2} when Set1 :: set(Element), Set2 :: set(Element).

Returns {Element, Set2}, where Element is the smallest element in Set1, and Set2 is this set with Element deleted.

Assumes that Set1 is not empty.

Examples

1> S0 = gb_sets:from_list([a,b,c]).
2> {Smallest,S1} = gb_sets:take_smallest(S0).
3> Smallest.
a
4> gb_sets:to_list(S1).
[b,c]

to_list(Set)

-spec to_list(Set) -> List when Set :: set(Element), List :: [Element].

Returns the elements of Set as an ordered list.

1> gb_sets:to_list(gb_sets:from_list([4,3,5,1,2])).
[1,2,3,4,5]

union(SetList)

-spec union(SetList) -> Set when SetList :: [set(Element), ...], Set :: set(Element).

Returns the union of a list of sets.

The union of multiple sets is a new set that contains all the elements from all sets, without duplicates.

Examples

1> S0 = gb_sets:from_list([a,b,c,d]).
2> S1 = gb_sets:from_list([d,e,f]).
3> S2 = gb_sets:from_list([q,r])
4> Sets = [S0, S1, S2].
5> Union = gb_sets:union(Sets).
6> gb_sets:to_list(Union).
[a,b,c,d,e,f,q,r]

union(Set1, Set2)

-spec union(Set1, Set2) -> Set3 when Set1 :: set(Element), Set2 :: set(Element), Set3 :: set(Element).

Returns the union of Set1 and Set2.

The union of two sets is a new set that contains all the elements from both sets, without duplicates.

Examples

1> S0 = gb_sets:from_list([a,b,c,d]).
2> S1 = gb_sets:from_list([c,d,e,f]).
3> Union = gb_sets:union(S0, S1).
4> gb_sets:to_list(Union).
[a,b,c,d,e,f]

gb_trees

General balanced trees.

This module provides Prof. Arne Andersson's General Balanced Trees. These have no storage overhead compared to unbalanced binary trees, and their performance is better than AVL trees.

This module considers two keys as different if and only if they do not compare equal (==).

Data Structure

Trees and iterators are built using opaque data structures that should not be pattern-matched from outside this module.

There is no attempt to balance trees after deletions. As deletions do not increase the height of a tree, this should be OK.

The original balance condition h(T) <= ceil(c * log(|T|)) has been changed to the similar (but not quite equivalent) condition 2 ^ h(T) <= |T| ^ c. This should also be OK.

See Also

dict, gb_sets

Summary

Types

iter()

iter(Key, Value)

A general balanced tree iterator.

tree()

tree(Key, Value)

A general balanced tree.

Functions

balance(Tree1)

Rebalances Tree1.

delete(Key, Tree1)

Removes the node with key Key from Tree1 and returns the new tree. Assumes that the key is present in the tree, crashes otherwise.

delete_any(Key, Tree1)

Removes the node with key Key from Tree1 if the key is present in the tree, otherwise does nothing. Returns the new tree.

empty()

Returns a new empty tree.

enter(Key, Value, Tree1)

Inserts Key with value Value into Tree1 if the key is not present in the tree, otherwise updates Key to value Value in Tree1. Returns the new tree.

from_orddict(List)

Turns an ordered list List of key-value tuples into a tree. The list must not contain duplicate keys.

get(Key, Tree)

Retrieves the value stored with Key in Tree. Assumes that the key is present in the tree, crashes otherwise.

insert(Key, Value, Tree1)

Inserts Key with value Value into Tree1 and returns the new tree. Assumes that the key is not present in the tree, crashes otherwise.

is_defined(Key, Tree)

Returns true if Key is present in Tree, otherwise false.

is_empty(Tree)

Returns true if Tree is an empty tree, othwewise false.

iterator(Tree)

Returns an iterator that can be used for traversing the entries of Tree; see next/1.

iterator(Tree, Order)

Returns an iterator that can be used for traversing the entries of Tree in either ordered or reversed direction; see next/1.

iterator_from(Key, Tree)

Returns an iterator that can be used for traversing the entries of Tree; see next/1. The difference as compared to the iterator returned by iterator/1 is that the iterator starts with the first key greater than or equal to Key.

iterator_from(Key, Tree, Order)

Returns an iterator that can be used for traversing the entries of Tree in either ordered or reversed direction; see next/1. The difference as compared to the iterator returned by iterator/2 is that the iterator starts with the first key next to or equal to Key.

keys(Tree)

Returns the keys in Tree as an ordered list.

larger(Key1, Tree)

Returns {Key2, Value}, where Key2 is the least key strictly greater than Key1, Value is the value associated with this key.

largest(Tree)

Returns {Key, Value}, where Key is the largest key in Tree, and Value is the value associated with this key. Assumes that the tree is not empty.

lookup(Key, Tree)

Looks up Key in Tree. Returns {value, Value}, or none if Key is not present.

map(Function, Tree1)

Maps function F(K, V1) -> V2 to all key-value pairs of tree Tree1. Returns a new tree Tree2 with the same set of keys as Tree1 and the new set of values V2.

next(Iter1)

Returns {Key, Value, Iter2}, where Key is the next key referred to by iterator Iter1, and Iter2 is the new iterator to be used for traversing the remaining nodes, or the atom none if no nodes remain.

size(Tree)

Returns the number of nodes in Tree.

smaller(Key1, Tree)

Returns {Key2, Value}, where Key2 is the greatest key strictly less than Key1, Value is the value associated with this key.

smallest(Tree)

Returns {Key, Value}, where Key is the smallest key in Tree, and Value is the value associated with this key. Assumes that the tree is not empty.

take(Key, Tree1)

Returns a value Value from node with key Key and new Tree2 without the node with this value. Assumes that the node with key is present in the tree, crashes otherwise.

take_any(Key, Tree1)

Returns a value Value from node with key Key and new Tree2 without the node with this value. Returns error if the node with the key is not present in the tree.

take_largest(Tree1)

Returns {Key, Value, Tree2}, where Key is the largest key in Tree1, Value is the value associated with this key, and Tree2 is this tree with the corresponding node deleted. Assumes that the tree is not empty.

take_smallest(Tree1)

Returns {Key, Value, Tree2}, where Key is the smallest key in Tree1, Value is the value associated with this key, and Tree2 is this tree with the corresponding node deleted. Assumes that the tree is not empty.

to_list(Tree)

Converts a tree into an ordered list of key-value tuples.

update(Key, Value, Tree1)

Updates Key to value Value in Tree1 and returns the new tree. Assumes that the key is present in the tree.

values(Tree)

Returns the values in Tree as an ordered list, sorted by their corresponding keys. Duplicates are not removed.

Types

iter()

-type iter() :: iter(_, _).

iter(Key, Value)

-opaque iter(Key, Value)

A general balanced tree iterator.

tree()

-type tree() :: tree(_, _).

tree(Key, Value)

-opaque tree(Key, Value)

A general balanced tree.

Functions

balance(Tree1)

-spec balance(Tree1) -> Tree2 when Tree1 :: tree(Key, Value), Tree2 :: tree(Key, Value).

Rebalances Tree1.

Notice that this is rarely necessary, but can be motivated when many nodes have been deleted from the tree without further insertions. Rebalancing can then be forced to minimize lookup times, as deletion does not rebalance the tree.

delete(Key, Tree1)

-spec delete(Key, Tree1) -> Tree2 when Tree1 :: tree(Key, Value), Tree2 :: tree(Key, Value).

Removes the node with key Key from Tree1 and returns the new tree. Assumes that the key is present in the tree, crashes otherwise.

delete_any(Key, Tree1)

-spec delete_any(Key, Tree1) -> Tree2 when Tree1 :: tree(Key, Value), Tree2 :: tree(Key, Value).

Removes the node with key Key from Tree1 if the key is present in the tree, otherwise does nothing. Returns the new tree.

empty()

-spec empty() -> tree(none(), none()).

Returns a new empty tree.

enter(Key, Value, Tree1)

-spec enter(Key, Value, Tree1) -> Tree2 when Tree1 :: tree(Key, Value), Tree2 :: tree(Key, Value).

Inserts Key with value Value into Tree1 if the key is not present in the tree, otherwise updates Key to value Value in Tree1. Returns the new tree.

from_orddict(List)

-spec from_orddict(List) -> Tree when List :: [{Key, Value}], Tree :: tree(Key, Value).

Turns an ordered list List of key-value tuples into a tree. The list must not contain duplicate keys.

get(Key, Tree)

-spec get(Key, Tree) -> Value when Tree :: tree(Key, Value).

Retrieves the value stored with Key in Tree. Assumes that the key is present in the tree, crashes otherwise.

insert(Key, Value, Tree1)

-spec insert(Key, Value, Tree1) -> Tree2 when Tree1 :: tree(Key, Value), Tree2 :: tree(Key, Value).

Inserts Key with value Value into Tree1 and returns the new tree. Assumes that the key is not present in the tree, crashes otherwise.

is_defined(Key, Tree)

-spec is_defined(Key, Tree) -> boolean() when Tree :: tree(Key, Value :: term()).

Returns true if Key is present in Tree, otherwise false.

is_empty(Tree)

-spec is_empty(Tree) -> boolean() when Tree :: tree().

Returns true if Tree is an empty tree, othwewise false.

iterator(Tree)

-spec iterator(Tree) -> Iter when Tree :: tree(Key, Value), Iter :: iter(Key, Value).

Returns an iterator that can be used for traversing the entries of Tree; see next/1.

Equivalent to iterator(Tree, ordered).

iterator(Tree, Order)

(since OTP 27.0)

-spec iterator(Tree, Order) -> Iter
                  when Tree :: tree(Key, Value), Iter :: iter(Key, Value), Order :: ordered | reversed.

Returns an iterator that can be used for traversing the entries of Tree in either ordered or reversed direction; see next/1.

The implementation of this is very efficient; traversing the whole tree using next/1 is only slightly slower than getting the list of all elements using to_list/1 and traversing that. The main advantage of the iterator approach is that it does not require the complete list of all elements to be built in memory at one time.

iterator_from(Key, Tree)

(since OTP 18.0)

-spec iterator_from(Key, Tree) -> Iter when Tree :: tree(Key, Value), Iter :: iter(Key, Value).

Returns an iterator that can be used for traversing the entries of Tree; see next/1. The difference as compared to the iterator returned by iterator/1 is that the iterator starts with the first key greater than or equal to Key.

Equivalent to iterator_from(Key, Tree, ordered).

iterator_from(Key, Tree, Order)

(since OTP 27.0)

-spec iterator_from(Key, Tree, Order) -> Iter
                       when
                           Tree :: tree(Key, Value),
                           Iter :: iter(Key, Value),
                           Order :: ordered | reversed.

Returns an iterator that can be used for traversing the entries of Tree in either ordered or reversed direction; see next/1. The difference as compared to the iterator returned by iterator/2 is that the iterator starts with the first key next to or equal to Key.

keys(Tree)

-spec keys(Tree) -> [Key] when Tree :: tree(Key, Value :: term()).

Returns the keys in Tree as an ordered list.

larger(Key1, Tree)

(since OTP 27.0)

-spec larger(Key1, Tree) -> none | {Key2, Value} when Key1 :: Key, Key2 :: Key, Tree :: tree(Key, Value).

Returns {Key2, Value}, where Key2 is the least key strictly greater than Key1, Value is the value associated with this key.

Returns none if no such pair exists.

largest(Tree)

-spec largest(Tree) -> {Key, Value} when Tree :: tree(Key, Value).

Returns {Key, Value}, where Key is the largest key in Tree, and Value is the value associated with this key. Assumes that the tree is not empty.

lookup(Key, Tree)

-spec lookup(Key, Tree) -> none | {value, Value} when Tree :: tree(Key, Value).

Looks up Key in Tree. Returns {value, Value}, or none if Key is not present.

map(Function, Tree1)

-spec map(Function, Tree1) -> Tree2
             when
                 Function :: fun((K :: Key, V1 :: Value1) -> V2 :: Value2),
                 Tree1 :: tree(Key, Value1),
                 Tree2 :: tree(Key, Value2).

Maps function F(K, V1) -> V2 to all key-value pairs of tree Tree1. Returns a new tree Tree2 with the same set of keys as Tree1 and the new set of values V2.

next(Iter1)

-spec next(Iter1) -> none | {Key, Value, Iter2}
              when Iter1 :: iter(Key, Value), Iter2 :: iter(Key, Value).

Returns {Key, Value, Iter2}, where Key is the next key referred to by iterator Iter1, and Iter2 is the new iterator to be used for traversing the remaining nodes, or the atom none if no nodes remain.

size(Tree)

-spec size(Tree) -> non_neg_integer() when Tree :: tree().

Returns the number of nodes in Tree.

smaller(Key1, Tree)

(since OTP 27.0)

-spec smaller(Key1, Tree) -> none | {Key2, Value}
                 when Key1 :: Key, Key2 :: Key, Tree :: tree(Key, Value).

Returns {Key2, Value}, where Key2 is the greatest key strictly less than Key1, Value is the value associated with this key.

Returns none if no such pair exists.

smallest(Tree)

-spec smallest(Tree) -> {Key, Value} when Tree :: tree(Key, Value).

Returns {Key, Value}, where Key is the smallest key in Tree, and Value is the value associated with this key. Assumes that the tree is not empty.

take(Key, Tree1)

(since OTP 20.0)

-spec take(Key, Tree1) -> {Value, Tree2}
              when Tree1 :: tree(Key, _), Tree2 :: tree(Key, _), Key :: term(), Value :: term().

Returns a value Value from node with key Key and new Tree2 without the node with this value. Assumes that the node with key is present in the tree, crashes otherwise.

take_any(Key, Tree1)

(since OTP 20.0)

-spec take_any(Key, Tree1) -> {Value, Tree2} | error
                  when Tree1 :: tree(Key, _), Tree2 :: tree(Key, _), Key :: term(), Value :: term().

Returns a value Value from node with key Key and new Tree2 without the node with this value. Returns error if the node with the key is not present in the tree.

take_largest(Tree1)

-spec take_largest(Tree1) -> {Key, Value, Tree2}
                      when Tree1 :: tree(Key, Value), Tree2 :: tree(Key, Value).

Returns {Key, Value, Tree2}, where Key is the largest key in Tree1, Value is the value associated with this key, and Tree2 is this tree with the corresponding node deleted. Assumes that the tree is not empty.

take_smallest(Tree1)

-spec take_smallest(Tree1) -> {Key, Value, Tree2}
                       when Tree1 :: tree(Key, Value), Tree2 :: tree(Key, Value).

Returns {Key, Value, Tree2}, where Key is the smallest key in Tree1, Value is the value associated with this key, and Tree2 is this tree with the corresponding node deleted. Assumes that the tree is not empty.

to_list(Tree)

-spec to_list(Tree) -> [{Key, Value}] when Tree :: tree(Key, Value).

Converts a tree into an ordered list of key-value tuples.

update(Key, Value, Tree1)

-spec update(Key, Value, Tree1) -> Tree2 when Tree1 :: tree(Key, Value), Tree2 :: tree(Key, Value).

Updates Key to value Value in Tree1 and returns the new tree. Assumes that the key is present in the tree.

values(Tree)

-spec values(Tree) -> [Value] when Tree :: tree(Key :: term(), Value).

Returns the values in Tree as an ordered list, sorted by their corresponding keys. Duplicates are not removed.

json

A library for encoding and decoding JSON.

This module implements EEP68.

Both encoder and decoder fully conform to RFC 8259 and ECMA 404 standards. The decoder is tested using JSONTestSuite.

Summary

Types

array_finish_fun()

array_push_fun()

array_start_fun()

continuation_state()

decode_value()

decoders()

encode_map(Value)

encode_value()

Simple JSON value encodeable with json:encode/1.

encoder()

formatter()

from_binary_fun()

object_finish_fun()

object_push_fun()

object_start_fun()

Functions

decode(Binary)

Parses a JSON value from Binary.

decode(Binary, Acc0, Decoders)

Parses a JSON value from Binary.

decode_continue/2

Continue parsing a stream of bytes of a JSON value.

decode_start(Binary, Acc, Decoders)

Begin parsing a stream of bytes of a JSON value.

encode(Term)

Generates JSON corresponding to Term.

encode(Term, Encoder)

Generates JSON corresponding to Term.

encode_atom/2

Default encoder for atoms used by json:encode/1.

encode_binary(Bin)

Default encoder for binaries as JSON strings used by json:encode/1.

encode_binary_escape_all(Bin)

Encoder for binaries as JSON strings producing pure-ASCII JSON.

encode_float(Float)

Default encoder for floats as JSON numbers used by json:encode/1.

encode_integer(Integer)

Default encoder for integers as JSON numbers used by json:encode/1.

encode_key_value_list(List, Encode)

Encoder for lists of key-value pairs as JSON objects.

encode_key_value_list_checked(List, Encode)

Encoder for lists of key-value pairs as JSON objects.

encode_list(List, Encode)

Default encoder for lists as JSON arrays used by json:encode/1.

encode_map(Map, Encode)

Default encoder for maps as JSON objects used by json:encode/1.

encode_map_checked(Map, Encode)

Encoder for maps as JSON objects.

encode_value(Value, Encode)

Default encoder used by json:encode/1.

format(Term)

Generates formatted JSON corresponding to Term.

format/2

Generates formatted JSON corresponding to Term.

format(Term, Encoder, Options)

Generates formatted JSON corresponding to Term.

format_key_value_list/3

Format function for lists of key-value pairs as JSON objects.

format_key_value_list_checked(KVList, UserEnc, State)

Format function for lists of key-value pairs as JSON objects.

format_value(Value, Encode, State)

Default format function used by json:format/1.

Types

array_finish_fun()

(since OTP 27.0)

-type array_finish_fun() :: fun((ArrayAcc :: dynamic(), OldAcc :: dynamic()) -> {dynamic(), dynamic()}).

array_push_fun()

(since OTP 27.0)

-type array_push_fun() :: fun((Value :: dynamic(), Acc :: dynamic()) -> NewAcc :: dynamic()).

array_start_fun()

(since OTP 27.0)

-type array_start_fun() :: fun((Acc :: dynamic()) -> ArrayAcc :: dynamic()).

continuation_state()

(since OTP 27.0)

-opaque continuation_state()

decode_value()

(since OTP 27.0)

-type decode_value() ::
          integer() |
          float() |
          boolean() |
          null |
          binary() |
          [decode_value()] |
          #{binary() => decode_value()}.

decoders()

(since OTP 27.0)

-type decoders() ::
          #{array_start => array_start_fun(),
            array_push => array_push_fun(),
            array_finish => array_finish_fun(),
            object_start => object_start_fun(),
            object_push => object_push_fun(),
            object_finish => object_finish_fun(),
            float => from_binary_fun(),
            integer => from_binary_fun(),
            string => from_binary_fun(),
            null => term()}.

encode_map(Value)

(not exported) (since OTP 27.0)

-type encode_map(Value) :: #{binary() | atom() | integer() => Value}.

encode_value()

(since OTP 27.0)

-type encode_value() ::
          integer() |
          float() |
          boolean() |
          null |
          binary() |
          atom() |
          [encode_value()] |
          encode_map(encode_value()).

Simple JSON value encodeable with json:encode/1.

encoder()

(since OTP 27.0)

-type encoder() :: fun((dynamic(), encoder()) -> iodata()).

formatter()

(since OTP 27.0)

-type formatter() :: fun((Term :: dynamic(), Encoder :: formatter(), State :: map()) -> iodata()).

from_binary_fun()

(since OTP 27.0)

-type from_binary_fun() :: fun((binary()) -> dynamic()).

object_finish_fun()

(since OTP 27.0)

-type object_finish_fun() ::
          fun((ObjectAcc :: dynamic(), OldAcc :: dynamic()) -> {dynamic(), dynamic()}).

object_push_fun()

(since OTP 27.0)

-type object_push_fun() ::
          fun((Key :: dynamic(), Value :: dynamic(), Acc :: dynamic()) -> NewAcc :: dynamic()).

object_start_fun()

(since OTP 27.0)

-type object_start_fun() :: fun((Acc :: dynamic()) -> ObjectAcc :: dynamic()).

Functions

decode(Binary)

(since OTP 27.0)

-spec decode(binary()) -> decode_value().

Parses a JSON value from Binary.

Supports basic data mapping:

JSON | Erlang
Number | integer() | float()
Boolean | true | false
Null | null
String | binary()
Object | #{binary() => _}

Errors

error(unexpected_end) if Binary contains incomplete JSON value

error({invalid_byte, Byte}) if Binary contains unexpected byte or invalid UTF-8 byte

error({unexpected_sequence, Bytes}) if Binary contains invalid UTF-8 escape

Example

> json:decode(<<"{\"foo\": 1}">>).
#{<<"foo">> => 1}

decode(Binary, Acc0, Decoders)

(since OTP 27.0)

-spec decode(binary(), dynamic(), decoders()) -> {Result :: dynamic(), Acc :: dynamic(), binary()}.

Parses a JSON value from Binary.

Similar to decode/1 except the decoding process can be customized with the callbacks specified in Decoders. The callbacks will use the Acc value as the initial accumulator.

Any leftover, unparsed data in Binary will be returned.

Default callbacks

All callbacks are optional. If not provided, they will fall back to implementations used by the decode/1 function:

for array_start: fun(_) -> [] end

for array_push: fun(Elem, Acc) -> [Elem | Acc] end

for array_finish: fun(Acc, OldAcc) -> {lists:reverse(Acc), OldAcc} end

for object_start: fun(_) -> [] end

for object_push: fun(Key, Value, Acc) -> [{Key, Value} | Acc] end

for object_finish: fun(Acc, OldAcc) -> {maps:from_list(Acc), OldAcc} end

for float: fun erlang:binary_to_float/1

for integer: fun erlang:binary_to_integer/1

for string: fun (Value) -> Value end

for null: the atom null

Errors

error({invalid_byte, Byte}) if Binary contains unexpected byte or invalid UTF-8 byte

error({unexpected_sequence, Bytes}) if Binary contains invalid UTF-8 escape

error(unexpected_end) if Binary contains incomplete JSON value

Example

Decoding object keys as atoms:

> Push = fun(Key, Value, Acc) -> [{binary_to_existing_atom(Key), Value} | Acc] end.
> json:decode(<<"{\"foo\": 1}">>, ok, #{object_push => Push}).
{#{foo => 1},ok,<<>>}

decode_continue/2

(since OTP 27.0)

-spec decode_continue(binary() | end_of_input, Opaque :: term()) ->
                         {Result :: dynamic(), Acc :: dynamic(), binary()} |
                         {continue, continuation_state()}.

Continue parsing a stream of bytes of a JSON value.

Similar to decode_start/3, if the function returns {continue, State} and there is no more data, use end_of_input instead of a binary.

> {continue, State} = json:decode_start(<<"{\"foo\":">>, ok, #{}).
> json:decode_continue(<<"1}">>, State).
{#{foo => 1},ok,<<>>}

> {continue, State} = json:decode_start(<<"123">>, ok, #{}).
> json:decode_continue(end_of_input, State).
{123,ok,<<>>}

decode_start(Binary, Acc, Decoders)

(since OTP 27.0)

-spec decode_start(binary(), dynamic(), decoders()) ->
                      {Result :: dynamic(), Acc :: dynamic(), binary()} |
                      {continue, continuation_state()}.

Begin parsing a stream of bytes of a JSON value.

Similar to decode/3 but returns when a complete JSON value can be parsed or returns {continue, State} for incomplete data, the State can be fed to the decode_continue/2 function when more data is available.

encode(Term)

(since OTP 27.0)

-spec encode(encode_value()) -> iodata().

Generates JSON corresponding to Term.

Supports basic data mapping:

Erlang | JSON
integer() | float() | Number
true | false | Boolean
null | Null
binary() | String
atom() | String
list() | Array
#{binary() => _} | Object
#{atom() => _} | Object
#{integer() => _} | Object

This is equivalent to encode(Term, fun json:encode_value/2).

Examples

> iolist_to_binary(json:encode(#{foo => <<"bar">>})).
<<"{\"foo\":\"bar\"}">>

encode(Term, Encoder)

(since OTP 27.0)

-spec encode(dynamic(), encoder()) -> iodata().

Generates JSON corresponding to Term.

Can be customised with the Encoder callback. The callback will be recursively called for all the data to be encoded and is expected to return the corresponding encoded JSON as iodata.

Various encode_* functions in this module can be used to help in constructing such callbacks.

Examples

An encoder that uses a heuristic to differentiate object-like lists of key-value pairs from plain lists:

> encoder([{_, _} | _] = Value, Encode) -> json:encode_key_value_list(Value, Encode);
> encoder(Other, Encode) -> json:encode_value(Other, Encode).
> custom_encode(Value) -> json:encode(Value, fun(Value, Encode) -> encoder(Value, Encode) end).
> iolist_to_binary(custom_encode([{a, []}, {b, 1}])).
<<"{\"a\":[],\"b\":1}">>

encode_atom/2

(since OTP 27.0)

-spec encode_atom(atom(), encoder()) -> iodata().

Default encoder for atoms used by json:encode/1.

Encodes the atom null as JSON null, atoms true and false as JSON booleans, and everything else as JSON strings calling the Encode callback with the corresponding binary.

encode_binary(Bin)

(since OTP 27.0)

-spec encode_binary(binary()) -> iodata().

Default encoder for binaries as JSON strings used by json:encode/1.

Errors

error(unexpected_end) if the binary contains incomplete UTF-8 sequences.

error({invalid_byte, Byte}) if the binary contains invalid UTF-8 sequences.

encode_binary_escape_all(Bin)

(since OTP 27.0)

-spec encode_binary_escape_all(binary()) -> iodata().

Encoder for binaries as JSON strings producing pure-ASCII JSON.

For any non-ASCII unicode character, a corresponding \\uXXXX sequence is used.

Errors

error(unexpected_end) if the binary contains incomplete UTF-8 sequences.

error({invalid_byte, Byte}) if the binary contains invalid UTF-8 sequences.

encode_float(Float)

(since OTP 27.0)

-spec encode_float(float()) -> iodata().

Default encoder for floats as JSON numbers used by json:encode/1.

encode_integer(Integer)

(since OTP 27.0)

-spec encode_integer(integer()) -> iodata().

Default encoder for integers as JSON numbers used by json:encode/1.

encode_key_value_list(List, Encode)

(since OTP 27.0)

-spec encode_key_value_list([{term(), term()}], encoder()) -> iodata().

Encoder for lists of key-value pairs as JSON objects.

Accepts lists with atom, binary, integer, or float keys.

encode_key_value_list_checked(List, Encode)

(since OTP 27.0)

-spec encode_key_value_list_checked([{term(), term()}], encoder()) -> iodata().

Encoder for lists of key-value pairs as JSON objects.

Accepts lists with atom, binary, integer, or float keys. Verifies that no duplicate keys will be produced in the resulting JSON object.

Errors

Raises error({duplicate_key, Key}) if there are duplicates.

encode_list(List, Encode)

(since OTP 27.0)

-spec encode_list(list(), encoder()) -> iodata().

Default encoder for lists as JSON arrays used by json:encode/1.

encode_map(Map, Encode)

(since OTP 27.0)

-spec encode_map(encode_map(dynamic()), encoder()) -> iodata().

Default encoder for maps as JSON objects used by json:encode/1.

Accepts maps with atom, binary, integer, or float keys.

encode_map_checked(Map, Encode)

(since OTP 27.0)

-spec encode_map_checked(map(), encoder()) -> iodata().

Encoder for maps as JSON objects.

Accepts maps with atom, binary, integer, or float keys. Verifies that no duplicate keys will be produced in the resulting JSON object.

Errors

Raises error({duplicate_key, Key}) if there are duplicates.

encode_value(Value, Encode)

(since OTP 27.0)

-spec encode_value(dynamic(), encoder()) -> iodata().

Default encoder used by json:encode/1.

Recursively calls Encode on all the values in Value.

format(Term)

(since OTP 27.1)

-spec format(Term :: encode_value()) -> iodata().

Generates formatted JSON corresponding to Term.

Similiar to encode/1 but with added whitespaces for formatting.

> io:put_chars(json:format(#{foo => <<"bar">>, baz => 52})).
{
  "baz": 52,
  "foo": "bar"
}
ok

format/2

(since OTP 27.1)

-spec format(Term :: encode_value(), Opts :: map()) -> iodata();
            (Term :: dynamic(), Encoder :: formatter()) -> iodata().

Generates formatted JSON corresponding to Term.

Equivalent to format(Term, fun json:format_value/3, Options) or format(Term, Encoder, #{})

format(Term, Encoder, Options)

(since OTP 27.1)

-spec format(Term :: dynamic(), Encoder :: formatter(), Options :: map()) -> iodata().

Generates formatted JSON corresponding to Term.

Similar to encode/2, can be customised with the Encoder callback and Options.

Options can include 'indent' to specify number of spaces per level and 'max' which loosely limits the width of lists.

The Encoder will get a 'State' argument which contains the 'Options' maps merged with other data when recursing through 'Term'.

format_value/3 or various encode_* functions in this module can be used to help in constructing such callbacks.

> formatter({posix_time, SysTimeSecs}, Encode, State) ->
    TimeStr = calendar:system_time_to_rfc3339(SysTimeSecs, [{offset, "Z"}]),
    json:format_value(unicode:characters_to_binary(TimeStr), Encode, State);
> formatter(Other, Encode, State) -> json:format_value(Other, Encode, State).
>
> Fun = fun(Value, Encode, State) -> formatter(Value, Encode, State) end.
> Options = #{indent => 4}.
> Term = #{id => 1, time => {posix_time, erlang:system_time(seconds)}}.
>
> io:put_chars(json:format(Term, Fun, Options)).
{
    "id": 1,
    "time": "2024-05-23T16:07:48Z"
}
ok

format_key_value_list/3

(since OTP 27.2)

-spec format_key_value_list([{term(), term()}], Encode :: formatter(), State :: map()) -> iodata().

Format function for lists of key-value pairs as JSON objects.

Accepts lists with atom, binary, integer, or float keys.

format_key_value_list_checked(KVList, UserEnc, State)

(since OTP 27.2)

-spec format_key_value_list_checked([{term(), term()}], Encoder :: formatter(), State :: map()) ->
                                       iodata().

Format function for lists of key-value pairs as JSON objects.

Accepts lists with atom, binary, integer, or float keys. Verifies that no duplicate keys will be produced in the resulting JSON object.

Errors

Raises error({duplicate_key, Key}) if there are duplicates.

format_value(Value, Encode, State)

(since OTP 27.1)

-spec format_value(Value :: dynamic(), Encode :: formatter(), State :: map()) -> iodata().

Default format function used by json:format/1.

Recursively calls Encode on all the values in Value, and indents objects and lists.

orddict

Key-value dictionary as ordered list.

This module provides a Key-Value dictionary. An orddict is a representation of a dictionary, where a list of pairs is used to store the keys and values. The list is ordered after the keys in the Erlang term order.

This module provides the same interface as the dict module but with a defined representation. One difference is that while dict considers two keys as different if they do not match (=:=), this module considers two keys as different if and only if they do not compare equal (==).

Notes

Functions append/3 and append_list/3 are included so that keyed values can be stored in a list accumulator, for example:

> D0 = orddict:new(),
  D1 = orddict:store(files, [], D0),
  D2 = orddict:append(files, f1, D1),
  D3 = orddict:append(files, f2, D2),
  D4 = orddict:append(files, f3, D3),
  orddict:fetch(files, D4).
[f1,f2,f3]

This saves the trouble of first fetching a keyed value, appending a new value to the list of stored values, and storing the result.

Function fetch/2 is to be used if the key is known to be in the dictionary, otherwise function find/2.

See Also

dict, gb_trees

Summary

Types

orddict()

orddict(Key, Value)

Dictionary as returned by new/0.

Functions

append(Key, Value, Orddict1)

Appends a new Value to the current list of values associated with Key. An exception is generated if the initial value associated with Key is not a list of values.

append_list(Key, ValList, Orddict1)

Appends a list of values ValList to the current list of values associated with Key. An exception is generated if the initial value associated with Key is not a list of values.

erase(Key, Orddict1)

Erases all items with a specified key from a dictionary.

fetch(Key, Orddict)

Returns the value associated with Key in dictionary Orddict. This function assumes that the Key is present in the dictionary. An exception is generated if Key is not in the dictionary.

fetch_keys(Orddict)

Returns a list of all keys in a dictionary.

filter(Pred, Orddict1)

Orddict2 is a dictionary of all keys and values in Orddict1 for which Pred(Key, Value) is true.

find(Key, Orddict)

Searches for a key in a dictionary. Returns {ok, Value}, where Value is the value associated with Key, or error if the key is not present in the dictionary.

fold(Fun, Acc0, Orddict)

Calls Fun on successive keys and values of Orddict together with an extra argument Acc (short for accumulator). Fun must return a new accumulator that is passed to the next call. Acc0 is returned if the list is empty.

from_list(List)

Converts the Key-Value list List to a dictionary.

is_empty(Orddict)

Returns true if Orddict has no elements, otherwise false.

is_key(Key, Orddict)

Tests if Key is contained in dictionary Orddict.

map(Fun, Orddict1)

Calls Fun on successive keys and values of Orddict1 to return a new value for each key.

merge(Fun, Orddict1, Orddict2)

Merges two dictionaries, Orddict1 and Orddict2, to create a new dictionary. All the Key-Value pairs from both dictionaries are included in the new dictionary.

new()

Creates a new dictionary.

size(Orddict)

Returns the number of elements in an Orddict.

store(Key, Value, Orddict1)

Stores a Key-Value pair in a dictionary. If the Key already exists in Orddict1, the associated value is replaced by Value.

take(Key, Orddict)

This function returns value from dictionary and new dictionary without this value. Returns error if the key is not present in the dictionary.

to_list(Orddict)

Converts a dictionary to a list representation.

update(Key, Fun, Orddict1)

Updates a value in a dictionary by calling Fun on the value to get a new value. An exception is generated if Key is not present in the dictionary.

update(Key, Fun, Initial, Orddict1)

Updates a value in a dictionary by calling Fun on the value to get a new value. If Key is not present in the dictionary, Initial is stored as the first value.

update_counter(Key, Increment, Orddict1)

Adds Increment to the value associated with Key and store this value. If Key is not present in the dictionary, Increment is stored as the first value.

Types

orddict()

-type orddict() :: orddict(_, _).

orddict(Key, Value)

-type orddict(Key, Value) :: [{Key, Value}].

Dictionary as returned by new/0.

Functions

append(Key, Value, Orddict1)

-spec append(Key, Value, Orddict1) -> Orddict2
                when Orddict1 :: orddict(Key, Value), Orddict2 :: orddict(Key, Value).

Appends a new Value to the current list of values associated with Key. An exception is generated if the initial value associated with Key is not a list of values.

See also section Notes.

Example 1:

1> OrdDict1 = orddict:from_list([{x, []}]).
[{x,[]}]
2> OrdDict2 = orddict:append(x, 1, OrdDict1).
[{x,[1]}]
3> OrdDict3 = orddict:append(x, 2, OrdDict2).
[{x,[1,2]}]
4> orddict:append(y, 3, OrdDict3).
[{x,[1,2]},{y,[3]}]

Example 2:

1> OrdDict1 = orddict:from_list([{a, no_list}]).
[{a,no_list}]
2> orddict:append(a, 1, OrdDict1).
** exception error: bad argument
     in operator  ++/2
        called as no_list ++ [1]

append_list(Key, ValList, Orddict1)

-spec append_list(Key, ValList, Orddict1) -> Orddict2
                     when
                         ValList :: [Value],
                         Orddict1 :: orddict(Key, Value),
                         Orddict2 :: orddict(Key, Value).

Appends a list of values ValList to the current list of values associated with Key. An exception is generated if the initial value associated with Key is not a list of values.

See also section Notes.

Example:

1> OrdDict1 = orddict:from_list([{x, []}]).
[{x,[]}]
2> OrdDict2 = orddict:append_list(x, [1,2], OrdDict1).
[{x,[1,2]}]
3> OrdDict3 = orddict:append_list(y, [3,4], OrdDict2).
[{x,[1,2]},{y,[3,4]}]

erase(Key, Orddict1)

-spec erase(Key, Orddict1) -> Orddict2
               when Orddict1 :: orddict(Key, Value), Orddict2 :: orddict(Key, Value).

Erases all items with a specified key from a dictionary.

Example:

1> OrdDict1 = orddict:from_list([{a, 1}, {b, 2}]).
[{a,1},{b,2}]
2> orddict:erase(a, OrdDict1).
[{b,2}]

fetch(Key, Orddict)

-spec fetch(Key, Orddict) -> Value when Orddict :: orddict(Key, Value).

Returns the value associated with Key in dictionary Orddict. This function assumes that the Key is present in the dictionary. An exception is generated if Key is not in the dictionary.

See also section Notes.

Example:

1> OrdDict1 = orddict:from_list([{a, 1}, {b, 2}]).
[{a,1},{b,2}]
2> orddict:fetch(a, OrdDict1).
1
3> orddict:fetch(missing, OrdDict1).
** exception error: no function clause matching orddict:fetch(missing,[])

fetch_keys(Orddict)

-spec fetch_keys(Orddict) -> Keys when Orddict :: orddict(Key, Value :: term()), Keys :: [Key].

Returns a list of all keys in a dictionary.

Example:

1> OrdDict1 = orddict:from_list([{a, 1}, {b, 2}]).
[{a,1},{b,2}]
2> orddict:fetch_keys(OrdDict1).
[a,b]

filter(Pred, Orddict1)

-spec filter(Pred, Orddict1) -> Orddict2
                when
                    Pred :: fun((Key, Value) -> boolean()),
                    Orddict1 :: orddict(Key, Value),
                    Orddict2 :: orddict(Key, Value).

Orddict2 is a dictionary of all keys and values in Orddict1 for which Pred(Key, Value) is true.

Example:

1> OrdDict1 = orddict:from_list([{a, 1}, {b, 2}]).
[{a,1},{b,2}]
2> orddict:filter(fun (K, V) -> V > 1 end, OrdDict1).
[{b,2}]

find(Key, Orddict)

-spec find(Key, Orddict) -> {ok, Value} | error when Orddict :: orddict(Key, Value).

Searches for a key in a dictionary. Returns {ok, Value}, where Value is the value associated with Key, or error if the key is not present in the dictionary.

See also section Notes.

Example:

1> OrdDict1 = orddict:from_list([{a, 1}, {b, 2}]).
[{a,1},{b,2}]
2> orddict:find(a, OrdDict1).
{ok,1}
3> orddict:find(c, OrdDict1).
error

fold(Fun, Acc0, Orddict)

-spec fold(Fun, Acc0, Orddict) -> Acc1
              when
                  Fun :: fun((Key, Value, AccIn) -> AccOut),
                  Orddict :: orddict(Key, Value),
                  Acc0 :: Acc,
                  Acc1 :: Acc,
                  AccIn :: Acc,
                  AccOut :: Acc.

Calls Fun on successive keys and values of Orddict together with an extra argument Acc (short for accumulator). Fun must return a new accumulator that is passed to the next call. Acc0 is returned if the list is empty.

Example:

1> OrdDict1 = orddict:from_list([{a, 1}, {b, 2}]).
[{a,1},{b,2}]
2> orddict:fold(fun (K, V, Acc) -> [{K, V+100} | Acc] end, [], OrdDict1).
[{b,102},{a,101}]

from_list(List)

-spec from_list(List) -> Orddict when List :: [{Key, Value}], Orddict :: orddict(Key, Value).

Converts the Key-Value list List to a dictionary.

is_empty(Orddict)

(since OTP 17.0)

-spec is_empty(Orddict) -> boolean() when Orddict :: orddict().

Returns true if Orddict has no elements, otherwise false.

is_key(Key, Orddict)

-spec is_key(Key, Orddict) -> boolean() when Orddict :: orddict(Key, Value :: term()).

Tests if Key is contained in dictionary Orddict.

map(Fun, Orddict1)

-spec map(Fun, Orddict1) -> Orddict2
             when
                 Fun :: fun((Key, Value1) -> Value2),
                 Orddict1 :: orddict(Key, Value1),
                 Orddict2 :: orddict(Key, Value2).

Calls Fun on successive keys and values of Orddict1 to return a new value for each key.

Example:

1> OrdDict1 = orddict:from_list([{a, 1}, {b, 2}]).
[{a,1},{b,2}]
2> orddict:map(fun (_K, V) -> V + 100 end, OrdDict1).
[{a,101},{b,102}]

merge(Fun, Orddict1, Orddict2)

-spec merge(Fun, Orddict1, Orddict2) -> Orddict3
               when
                   Fun :: fun((Key, Value1, Value2) -> Value),
                   Orddict1 :: orddict(Key, Value1),
                   Orddict2 :: orddict(Key, Value2),
                   Orddict3 :: orddict(Key, Value).

Merges two dictionaries, Orddict1 and Orddict2, to create a new dictionary. All the Key-Value pairs from both dictionaries are included in the new dictionary.

If a key occurs in both dictionaries, Fun is called with the key and both values to return a new value.

merge/3 can be defined as follows, but is faster:

merge(Fun, D1, D2) ->
    fold(fun (K, V1, D) ->
                 update(K, fun (V2) -> Fun(K, V1, V2) end, V1, D)
         end, D2, D1).

Example:

1> OrdDict1 = orddict:from_list([{a, 1}, {b, 2}]).
[{a,1},{b,2}]
2> OrdDict2 = orddict:from_list([{b, 7}, {c, 8}]).
[{b,7},{c,8}]
3> orddict:merge(fun (K, V1, V2) -> V1 * V2 end, OrdDict1, OrdDict2).
[{a,1},{b,14},{c,8}]

new()

-spec new() -> orddict(none(), none()).

Creates a new dictionary.

size(Orddict)

-spec size(Orddict) -> non_neg_integer() when Orddict :: orddict().

Returns the number of elements in an Orddict.

store(Key, Value, Orddict1)

-spec store(Key, Value, Orddict1) -> Orddict2
               when Orddict1 :: orddict(Key, Value), Orddict2 :: orddict(Key, Value).

Stores a Key-Value pair in a dictionary. If the Key already exists in Orddict1, the associated value is replaced by Value.

Example:

1> OrdDict1 = orddict:from_list([{a, 1}, {b, 2}]).
[{a,1},{b,2}]
2> orddict:store(a, 99, OrdDict1).
[{a,99},{b,2}]
3> orddict:store(c, 100, OrdDict1).
[{a,1},{b,2},{c,100}]

take(Key, Orddict)

(since OTP 20.0)

-spec take(Key, Orddict) -> {Value, Orddict1} | error
              when
                  Orddict :: orddict(Key, Value),
                  Orddict1 :: orddict(Key, Value),
                  Key :: term(),
                  Value :: term().

This function returns value from dictionary and new dictionary without this value. Returns error if the key is not present in the dictionary.

Example:

1> OrdDict1 = orddict:from_list([{a, 1}, {b, 2}]).
[{a,1},{b,2}]
2> orddict:take(a, OrdDict1).
{1,[{b,2}]}
3> orddict:take(missing, OrdDict1).
error

to_list(Orddict)

-spec to_list(Orddict) -> List when Orddict :: orddict(Key, Value), List :: [{Key, Value}].

Converts a dictionary to a list representation.

update(Key, Fun, Orddict1)

-spec update(Key, Fun, Orddict1) -> Orddict2
                when
                    Fun :: fun((Value1 :: Value) -> Value2 :: Value),
                    Orddict1 :: orddict(Key, Value),
                    Orddict2 :: orddict(Key, Value).

Updates a value in a dictionary by calling Fun on the value to get a new value. An exception is generated if Key is not present in the dictionary.

Example:

1> OrdDict1 = orddict:from_list([{a, 1}, {b, 2}]).
[{a,1},{b,2}]
2> orddict:update(a, fun (V) -> V + 100 end, OrdDict1).
[{a,101},{b,2}]

update(Key, Fun, Initial, Orddict1)

-spec update(Key, Fun, Initial, Orddict1) -> Orddict2
                when
                    Initial :: Value,
                    Fun :: fun((Value1 :: Value) -> Value2 :: Value),
                    Orddict1 :: orddict(Key, Value),
                    Orddict2 :: orddict(Key, Value).

Updates a value in a dictionary by calling Fun on the value to get a new value. If Key is not present in the dictionary, Initial is stored as the first value.

For example, append/3 can be defined as follows:

append(Key, Val, D) ->
    update(Key, fun (Old) -> Old ++ [Val] end, [Val], D).

Example 1:

1> OrdDict1 = orddict:from_list([{a, 1}, {b, 2}]).
[{a,1},{b,2}]
2> orddict:update(c, fun (V) -> V + 100 end, 99, OrdDict1).
[{a,1},{b,2},{c,99}]

Example 2:

1> OrdDict1 = orddict:from_list([{a, 1}, {b, 2}]).
[{a,1},{b,2}]
2> orddict:update(a, fun (V) -> V + 100 end, 99, OrdDict1).
[{a,101},{b,2}]

update_counter(Key, Increment, Orddict1)

-spec update_counter(Key, Increment, Orddict1) -> Orddict2
                        when
                            Orddict1 :: orddict(Key, Value),
                            Orddict2 :: orddict(Key, Value),
                            Increment :: number().

Adds Increment to the value associated with Key and store this value. If Key is not present in the dictionary, Increment is stored as the first value.

This can be defined as follows, but is faster:

update_counter(Key, Incr, D) ->
    update(Key, fun (Old) -> Old + Incr end, Incr, D).

ordsets

Functions for manipulating sets as ordered lists.

Sets are collections of elements with no duplicate elements. An ordset is a representation of a set, where an ordered list is used to store the elements of the set. An ordered list is more efficient than an unordered list. Elements are ordered according to the Erlang term order.

This module provides the same interface as the sets module but with a defined representation. One difference is that while sets considers two elements as different if they do not match (=:=), this module considers two elements as different if and only if they do not compare equal (==).

See the Compatibility Section in the sets module for more information about the compatibility of the different implementations of sets in the Standard Library.

See Also

gb_sets, sets

Summary

Types

ordset(T)

As returned by new/0.

Functions

add_element(Element, Ordset1)

Returns a new ordered set formed from Ordset1 with Element inserted.

del_element(Element, Ordset1)

Returns a copy of Ordset1 with Element removed.

filter(Pred, Ordset1)

Filters elements in Ordset1 using predicate function Pred.

filtermap(Fun, Ordset1)

Calls Fun(Elem) for each Elem of Ordset1 to update or remove elements from Ordset1.

fold(Function, Acc0, Ordset)

Folds Function over every element in Ordset and returns the final value of the accumulator.

from_list(List)

Returns an ordered set of the elements in List.

intersection(OrdsetList)

Returns the intersection of the non-empty list of sets.

intersection(Ordset1, Ordset2)

Returns the intersection of Ordset1 and Ordset2.

is_disjoint(Ordset1, Ordset2)

Returns true if Ordset1 and Ordset2 are disjoint; otherwise, returns false.

is_element(Element, Ordset)

Returns true if Element is an element of Ordset; otherwise, returns false.

is_empty(Ordset)

Returns true if Ordset is an empty set; otherwise, returns false.

is_equal(Ordset1, Ordset2)

Returns true if Ordset1 and Ordset2 are equal, that is, if every element of one set is also a member of the other set; otherwise, returns false.

is_set(Ordset)

Returns true if Ordset is an ordered set of elements; otherwise, returns false.

is_subset(Ordset1, Ordset2)

Returns true when every element of Ordset1 is also a member of Ordset2; otherwise, returns false.

map(Fun, Ordset1)

Maps elements in Ordset1 with mapping function Fun.

new()

Returns a new empty ordered set.

size(Ordset)

Returns the number of elements in Ordset.

subtract(Ordset1, Ordset2)

Returns a new ordered set containing the elements of Ordset1 that are not elements in Ordset2.

to_list(Ordset)

Returns the elements of Ordset as a list.

union(OrdsetList)

Returns the union of a list of sets.

union(Ordset1, Ordset2)

Returns the union of Ordset1 and Ordset2.

Types

ordset(T)

-type ordset(T) :: [T].

As returned by new/0.

Functions

add_element(Element, Ordset1)

-spec add_element(Element, Ordset1) -> Ordset2
                     when Element :: E, Ordset1 :: ordset(T), Ordset2 :: ordset(T | E).

Returns a new ordered set formed from Ordset1 with Element inserted.

Examples

1> S0 = ordsets:new().
[]
2> S1 = ordsets:add_element(7, S0).
[7]
3> S2 = ordsets:add_element(42, S1).
[7,42]
4> ordsets:add_element(42, S2).
[7,42]

del_element(Element, Ordset1)

-spec del_element(Element, Ordset1) -> Ordset2
                     when Element :: term(), Ordset1 :: ordset(T), Ordset2 :: ordset(T).

Returns a copy of Ordset1 with Element removed.

Examples

1> S = ordsets:from_list([a,b,c]).
2> ordsets:del_element(c, S).
[a,b]
3> ordsets:del_element(x, S).
[a,b,c]

filter(Pred, Ordset1)

-spec filter(Pred, Ordset1) -> Ordset2
                when
                    Pred :: fun((Element :: T) -> boolean()), Ordset1 :: ordset(T), Ordset2 :: ordset(T).

Filters elements in Ordset1 using predicate function Pred.

Examples

1> S = ordsets:from_list([1,2,3,4,5,6,7]).
2> IsEven = fun(N) -> N rem 2 =:= 0 end.
3> ordsets:filter(IsEven, S).
[2,4,6]

filtermap(Fun, Ordset1)

(since OTP 27.0)

-spec filtermap(Fun, Ordset1) -> Ordset2
                   when
                       Fun :: fun((Element1 :: T1) -> boolean | {true, Element2 :: T2}),
                       Ordset1 :: ordset(T1),
                       Ordset2 :: ordset(T1 | T2).

Calls Fun(Elem) for each Elem of Ordset1 to update or remove elements from Ordset1.

Fun/1 must return either a Boolean or a tuple {true, Value}. The function returns the set of elements for which Fun returns a new value, with true being equivalent to {true, Elem}.

ordsets:filtermap/2 behaves as if it were defined as follows:

filtermap(Fun, Ordset1) ->
    ordsets:from_list(lists:filtermap(Fun, Ordset1)).

Examples

1> S = ordsets:from_list([2,4,5,6,8,9])
2> F = fun(X) ->
           case X rem 2 of
               0 -> {true, X div 2};
               1 -> false
           end
        end.
3> ordsets:filtermap(F, S).
[1,2,3,4]

fold(Function, Acc0, Ordset)

-spec fold(Function, Acc0, Ordset) -> Acc1
              when
                  Function :: fun((Element :: T, AccIn :: term()) -> AccOut :: term()),
                  Ordset :: ordset(T),
                  Acc0 :: term(),
                  Acc1 :: term().

Folds Function over every element in Ordset and returns the final value of the accumulator.

Examples

1> S = ordsets:from_list([1,2,3,4]).
2> Plus = fun erlang:'+'/2.
3> ordsets:fold(Plus, 0, S).
10

from_list(List)

-spec from_list(List) -> Ordset when List :: [T], Ordset :: ordset(T).

Returns an ordered set of the elements in List.

Examples

1> ordsets:from_list([a,b,a,b,b,c]).
[a,b,c]

intersection(OrdsetList)

-spec intersection(OrdsetList) -> Ordset when OrdsetList :: [ordset(_), ...], Ordset :: ordset(_).

Returns the intersection of the non-empty list of sets.

The intersection of multiple sets is a new set that contains only the elements that are present in all sets.

Examples

1> S0 = ordsets:from_list([a,b,c,d]).
2> S1 = ordsets:from_list([d,e,f]).
3> S2 = ordsets:from_list([q,r])
4> Sets = [S0, S1, S2].
5> ordsets:intersection([S0, S1, S2]).
[]
6> ordsets:intersection([S0, S1]).
[d]
7> ordsets:intersection([]).
** exception error: no function clause matching ordsets:intersection([])

intersection(Ordset1, Ordset2)

-spec intersection(Ordset1, Ordset2) -> Ordset3
                      when Ordset1 :: ordset(_), Ordset2 :: ordset(_), Ordset3 :: ordset(_).

Returns the intersection of Ordset1 and Ordset2.

The intersection of two sets is a new set that contains only the elements that are present in both sets.

Examples

1> S0 = ordsets:from_list([a,b,c,d]).
2> S1 = ordsets:from_list([c,d,e,f]).
3> S2 = ordsets:from_list([q,r]).
4> ordsets:intersection(S0, S1).
[c,d]
5> ordsets:intersection(S1, S2).
[]

is_disjoint(Ordset1, Ordset2)

-spec is_disjoint(Ordset1, Ordset2) -> boolean() when Ordset1 :: ordset(_), Ordset2 :: ordset(_).

Returns true if Ordset1 and Ordset2 are disjoint; otherwise, returns false.

Two sets are disjoint if they have no elements in common.

This function is equivalent to ordsets:intersection(Ordset1, Ordset2) =:= [], but faster.

Examples

1> S0 = ordsets:from_list([a,b,c,d]).
2> S1 = ordsets:from_list([d,e,f]).
3> S2 = ordsets:from_list([q,r])
4> ordsets:is_disjoint(S0, S1).
false
5> ordsets:is_disjoint(S1, S2).
true

is_element(Element, Ordset)

-spec is_element(Element, Ordset) -> boolean() when Element :: term(), Ordset :: ordset(_).

Returns true if Element is an element of Ordset; otherwise, returns false.

Examples

1> S = ordsets:from_list([a,b,c]).
2> ordsets:is_element(42, S).
false
3> ordsets:is_element(b, S).
true

is_empty(Ordset)

(since OTP 21.0)

-spec is_empty(Ordset) -> boolean() when Ordset :: ordset(_).

Returns true if Ordset is an empty set; otherwise, returns false.

Examples

1> ordsets:is_empty(ordsets:new()).
true
2> ordsets:is_empty(ordsets:from_list([1])).
false

is_equal(Ordset1, Ordset2)

(since OTP 27.0)

-spec is_equal(Ordset1, Ordset2) -> boolean() when Ordset1 :: ordset(_), Ordset2 :: ordset(_).

Returns true if Ordset1 and Ordset2 are equal, that is, if every element of one set is also a member of the other set; otherwise, returns false.

Examples

1> Empty = ordsets:new().
2> S = ordsets:from_list([a,b]).
3> ordsets:is_equal(S, S)
true
4> ordsets:is_equal(S, Empty)
false

is_set(Ordset)

-spec is_set(Ordset) -> boolean() when Ordset :: term().

Returns true if Ordset is an ordered set of elements; otherwise, returns false.

Note

This function returns true for any ordered list, even if it was not constructed using the functions in this module.

Examples

1> ordsets:is_set(ordsets:from_list([a,x,13,{p,q}])).
true
2> ordsets:is_set([a,b,c]).
true
3> ordsets:is_set([z,a]).
false
4> ordsets:is_set({a,b}).
false

is_subset(Ordset1, Ordset2)

-spec is_subset(Ordset1, Ordset2) -> boolean() when Ordset1 :: ordset(_), Ordset2 :: ordset(_).

Returns true when every element of Ordset1 is also a member of Ordset2; otherwise, returns false.

Examples

1> S0 = ordsets:from_list([a,b,c,d]).
2> S1 = ordsets:from_list([c,d]).
3> ordsets:is_subset(S1, S0).
true
4> ordsets:is_subset(S0, S1).
false
5> ordsets:is_subset(S0, S0).
true

map(Fun, Ordset1)

(since OTP 27.0)

-spec map(Fun, Ordset1) -> Ordset2
             when
                 Fun :: fun((Element1 :: T1) -> Element2 :: T2),
                 Ordset1 :: ordset(T1),
                 Ordset2 :: ordset(T2).

Maps elements in Ordset1 with mapping function Fun.

Examples

1> S = ordsets:from_list([1,2,3,4,5,6,7]).
2> F = fun(N) -> N div 2 end.
3> ordsets:map(F, S).
[0,1,2,3]

new()

-spec new() -> [].

Returns a new empty ordered set.

Examples

1> ordsets:new()
[]

size(Ordset)

-spec size(Ordset) -> non_neg_integer() when Ordset :: ordset(_).

Returns the number of elements in Ordset.

Examples

1> ordsets:size(ordsets:new()).
0
2> ordsets:size(ordsets:from_list([4,5,6])).
3

subtract(Ordset1, Ordset2)

-spec subtract(Ordset1, Ordset2) -> Ordset3
                  when Ordset1 :: ordset(_), Ordset2 :: ordset(_), Ordset3 :: ordset(_).

Returns a new ordered set containing the elements of Ordset1 that are not elements in Ordset2.

Examples

1> S0 = ordsets:from_list([a,b,c,d]).
2> S1 = ordsets:from_list([c,d,e,f]).
3> ordsets:subtract(S0, S1).
[a,b]
4> ordsets:subtract(S1, S0).
[e,f]

to_list(Ordset)

-spec to_list(Ordset) -> List when Ordset :: ordset(T), List :: [T].

Returns the elements of Ordset as a list.

Examples

1> S = ordsets:from_list([a,b]).
2> ordsets:to_list(S).
[a,b]

union(OrdsetList)

-spec union(OrdsetList) -> Ordset when OrdsetList :: [ordset(T)], Ordset :: ordset(T).

Returns the union of a list of sets.

The union of multiple sets is a new set that contains all the elements from all sets, without duplicates.

Examples

1> S0 = ordsets:from_list([a,b,c,d]).
2> S1 = ordsets:from_list([d,e,f]).
3> S2 = ordsets:from_list([q,r])
4> Sets = [S0, S1, S2].
5> ordsets:union(Sets).
[a,b,c,d,e,f,q,r]

union(Ordset1, Ordset2)

-spec union(Ordset1, Ordset2) -> Ordset3
               when Ordset1 :: ordset(T1), Ordset2 :: ordset(T2), Ordset3 :: ordset(T1 | T2).

Returns the union of Ordset1 and Ordset2.

The union of two sets is a new set that contains all the elements from both sets, without duplicates.

Examples

1> S0 = ordsets:from_list([a,b,c,d]).
2> S1 = ordsets:from_list([c,d,e,f]).
3> ordsets:union(S0, S1).
[a,b,c,d,e,f]

proplists

Support functions for property lists.

Property lists are ordinary lists containing entries in the form of either tuples, whose first elements are keys used for lookup and insertion, or atoms, which work as shorthand for tuples {Atom, true}. (Other terms are allowed in the lists, but are ignored by this module.) If there is more than one entry in a list for a certain key, the first occurrence normally overrides any later (irrespective of the arity of the tuples).

Property lists are useful for representing inherited properties, such as options passed to a function where a user can specify options overriding the default settings, object properties, annotations, and so on.

Two keys are considered equal if they match (=:=). That is, numbers are compared literally rather than by value, so that, for example, 1 and 1.0 are different keys.

Summary

Types

property()

A property item within a list

proplist()

A list of property/0, also knows as a proplist.

Functions

append_values(Key, ListIn)

Similar to get_all_values/2, but each value is wrapped in a list unless it is already itself a list. The resulting list of lists is concatenated. This is often useful for "incremental" options.

compact(ListIn)

Minimizes the representation of all entries in the list. This is equivalent to [property(P) || P <- ListIn].

delete(Key, List)

Deletes all entries associated with Key from List.

expand(Expansions, ListIn)

Expands particular properties to corresponding sets of properties (or other terms).

from_map(Map)

Converts the map Map to a property list.

get_all_values(Key, List)

Similar to get_value/2, but returns the list of values for all entries {Key, Value} in List. If no such entry exists, the result is the empty list.

get_bool(Key, List)

Returns the value of a boolean key/value option. If lookup(Key, List) would yield {Key, true}, this function returns true, otherwise false.

get_keys(List)

Returns an unordered list of the keys used in List, not containing duplicates.

get_value(Key, List)

Equivalent to get_value(Key, List, undefined).

get_value(Key, List, Default)

Returns the value of a simple key/value property in List. If lookup(Key, List) would yield {Key, Value}, this function returns the corresponding Value, otherwise Default.

is_defined(Key, List)

Returns true if List contains at least one entry associated with Key, otherwise false.

lookup(Key, List)

Returns the first entry associated with Key in List, if one exists, otherwise returns none. For an atom A in the list, the tuple {A, true} is the entry associated with A.

lookup_all(Key, List)

Returns the list of all entries associated with Key in List. If no such entry exists, the result is the empty list.

normalize(ListIn, Stages)

Passes ListIn through a sequence of substitution/expansion stages. For an aliases operation, function substitute_aliases/2 is applied using the specified list of aliases

property(PropertyIn)

Creates a normal form (minimal) representation of a property. If PropertyIn is {Key, true}, where Key is an atom, Key is returned, otherwise the whole term PropertyIn is returned.

property(Key, Value)

Creates a normal form (minimal) representation of a simple key/value property. Returns Key if Value is true and Key is an atom, otherwise a tuple {Key, Value} is returned.

split(List, Keys)

Partitions List into a list of sublists and a remainder.

substitute_aliases(Aliases, ListIn)

Substitutes keys of properties. For each entry in ListIn, if it is associated with some key K1 such that {K1, K2} occurs in Aliases, the key of the entry is changed to K2. If the same K1 occurs more than once in Aliases, only the first occurrence is used.

substitute_negations(Negations, ListIn)

Substitutes keys of boolean-valued properties and simultaneously negates their values.

to_map(List)

Converts the property list List to a map.

to_map(List, Stages)

Converts the property list List to a map after applying the normalizations given in Stages.

unfold(ListIn)

Unfolds all occurrences of atoms in ListIn to tuples {Atom, true}.

Types

property()

-type property() :: atom() | tuple().

A property item within a list

proplist()

-type proplist() :: [property()].

A list of property/0, also knows as a proplist.

Functions

append_values(Key, ListIn)

-spec append_values(Key, ListIn) -> ListOut when Key :: term(), ListIn :: [term()], ListOut :: [term()].

Similar to get_all_values/2, but each value is wrapped in a list unless it is already itself a list. The resulting list of lists is concatenated. This is often useful for "incremental" options.

Example:

append_values(a, [{a, [1,2]}, {b, 0}, {a, 3}, {c, -1}, {a, [4]}])

returns:

[1,2,3,4]

compact(ListIn)

-spec compact(ListIn) -> ListOut when ListIn :: [property()], ListOut :: [property()].

Minimizes the representation of all entries in the list. This is equivalent to [property(P) || P <- ListIn].

See also property/1, unfold/1.

delete(Key, List)

-spec delete(Key, List) -> List when Key :: term(), List :: [term()].

Deletes all entries associated with Key from List.

expand(Expansions, ListIn)

-spec expand(Expansions, ListIn) -> ListOut
                when
                    Expansions :: [{Property :: property(), Expansion :: [term()]}],
                    ListIn :: [term()],
                    ListOut :: [term()].

Expands particular properties to corresponding sets of properties (or other terms).

For each pair {Property, Expansion} in Expansions: if E is the first entry in ListIn with the same key as Property, and E and Property have equivalent normal forms, then E is replaced with the terms in Expansion, and any following entries with the same key are deleted from ListIn.

For example, the following expressions all return [fie, bar, baz, fum]:

expand([{foo, [bar, baz]}], [fie, foo, fum])
expand([{{foo, true}, [bar, baz]}], [fie, foo, fum])
expand([{{foo, false}, [bar, baz]}], [fie, {foo, false}, fum])

However, no expansion is done in the following call because {foo, false} shadows foo:

expand([{{foo, true}, [bar, baz]}], [{foo, false}, fie, foo, fum])

Notice that if the original property term is to be preserved in the result when expanded, it must be included in the expansion list. The inserted terms are not expanded recursively. If Expansions contains more than one property with the same key, only the first occurrence is used.

See also normalize/2.

from_map(Map)

(since OTP 24.0)

-spec from_map(Map) -> List
                  when Map :: #{Key => Value}, List :: [{Key, Value}], Key :: term(), Value :: term().

Converts the map Map to a property list.

get_all_values(Key, List)

-spec get_all_values(Key, List) -> [term()] when Key :: term(), List :: [term()].

Similar to get_value/2, but returns the list of values for all entries {Key, Value} in List. If no such entry exists, the result is the empty list.

get_bool(Key, List)

-spec get_bool(Key, List) -> boolean() when Key :: term(), List :: [term()].

Returns the value of a boolean key/value option. If lookup(Key, List) would yield {Key, true}, this function returns true, otherwise false.

See also get_value/2, lookup/2.

get_keys(List)

-spec get_keys(List) -> [term()] when List :: [term()].

Returns an unordered list of the keys used in List, not containing duplicates.

get_value(Key, List)

-spec get_value(Key, List) -> term() when Key :: term(), List :: [term()].

Equivalent to get_value(Key, List, undefined).

get_value(Key, List, Default)

-spec get_value(Key, List, Default) -> term() when Key :: term(), List :: [term()], Default :: term().

Returns the value of a simple key/value property in List. If lookup(Key, List) would yield {Key, Value}, this function returns the corresponding Value, otherwise Default.

See also get_all_values/2, get_bool/2, get_value/2, lookup/2.

is_defined(Key, List)

-spec is_defined(Key, List) -> boolean() when Key :: term(), List :: [term()].

Returns true if List contains at least one entry associated with Key, otherwise false.

lookup(Key, List)

-spec lookup(Key, List) -> none | tuple() when Key :: term(), List :: [term()].

Returns the first entry associated with Key in List, if one exists, otherwise returns none. For an atom A in the list, the tuple {A, true} is the entry associated with A.

See also get_bool/2, get_value/2, lookup_all/2.

lookup_all(Key, List)

-spec lookup_all(Key, List) -> [tuple()] when Key :: term(), List :: [term()].

Returns the list of all entries associated with Key in List. If no such entry exists, the result is the empty list.

See also lookup/2.

normalize(ListIn, Stages)

-spec normalize(ListIn, Stages) -> ListOut
                   when
                       ListIn :: [term()],
                       Stages :: [Operation],
                       Operation :: {aliases, Aliases} | {negations, Negations} | {expand, Expansions},
                       Aliases :: [{Key, Key}],
                       Negations :: [{Key, Key}],
                       Expansions :: [{Property :: property(), Expansion :: [term()]}],
                       ListOut :: [term()].

Passes ListIn through a sequence of substitution/expansion stages. For an aliases operation, function substitute_aliases/2 is applied using the specified list of aliases:

For a negations operation, substitute_negations/2 is applied using the specified negation list.

For an expand operation, function expand/2 is applied using the specified list of expansions.

The final result is automatically compacted (compare compact/1).

Typically you want to substitute negations first, then aliases, then perform one or more expansions (sometimes you want to pre-expand particular entries before doing the main expansion). You might want to substitute negations and/or aliases repeatedly, to allow such forms in the right-hand side of aliases and expansion lists.

See also substitute_negations/2.

property(PropertyIn)

-spec property(PropertyIn) -> PropertyOut when PropertyIn :: property(), PropertyOut :: property().

Creates a normal form (minimal) representation of a property. If PropertyIn is {Key, true}, where Key is an atom, Key is returned, otherwise the whole term PropertyIn is returned.

See also property/2.

property(Key, Value)

-spec property(Key, Value) -> Property
                  when Key :: term(), Value :: term(), Property :: atom() | {term(), term()}.

Creates a normal form (minimal) representation of a simple key/value property. Returns Key if Value is true and Key is an atom, otherwise a tuple {Key, Value} is returned.

See also property/1.

split(List, Keys)

-spec split(List, Keys) -> {Lists, Rest}
               when List :: [term()], Keys :: [term()], Lists :: [[term()]], Rest :: [term()].

Partitions List into a list of sublists and a remainder.

Lists contains one sublist for each key in Keys, in the corresponding order. The relative order of the elements in each sublist is preserved from the original List. Rest contains the elements in List that are not associated with any of the specified keys, also with their original relative order preserved.

Example:

split([{c, 2}, {e, 1}, a, {c, 3, 4}, d, {b, 5}, b], [a, b, c])

returns:

{[[a], [{b, 5}, b],[{c, 2}, {c, 3, 4}]], [{e, 1}, d]}

substitute_aliases(Aliases, ListIn)

-spec substitute_aliases(Aliases, ListIn) -> ListOut
                            when
                                Aliases :: [{Key, Key}],
                                Key :: term(),
                                ListIn :: [term()],
                                ListOut :: [term()].

Substitutes keys of properties. For each entry in ListIn, if it is associated with some key K1 such that {K1, K2} occurs in Aliases, the key of the entry is changed to K2. If the same K1 occurs more than once in Aliases, only the first occurrence is used.

For example, substitute_aliases([{color, colour}], L) replaces all tuples {color, ...} in L with {colour, ...}, and all atoms color with colour.

See also normalize/2, substitute_negations/2.

substitute_negations(Negations, ListIn)

-spec substitute_negations(Negations, ListIn) -> ListOut
                              when
                                  Negations :: [{Key1, Key2}],
                                  Key1 :: term(),
                                  Key2 :: term(),
                                  ListIn :: [term()],
                                  ListOut :: [term()].

Substitutes keys of boolean-valued properties and simultaneously negates their values.

For each entry in ListIn, if it is associated with some key K1 such that {K1, K2} occurs in Negations: if the entry was {K1, true}, it is replaced with {K2, false}, otherwise with K2, thus changing the name of the option and simultaneously negating the value specified by get_bool(Key, ListIn). If the same K1 occurs more than once in Negations, only the first occurrence is used.

For example, substitute_negations([{no_foo, foo}], L) replaces any atom no_foo or tuple {no_foo, true} in L with {foo, false}, and any other tuple {no_foo, ...} with foo.

See also get_bool/2, normalize/2, substitute_aliases/2.

to_map(List)

(since OTP 24.0)

-spec to_map(List) -> Map
                when
                    List :: [Shorthand | {Key, Value} | term()],
                    Map :: #{Shorthand => true, Key => Value},
                    Shorthand :: atom(),
                    Key :: term(),
                    Value :: term().

Converts the property list List to a map.

Shorthand atom values in List will be expanded to an association of the form Atom => true. Tuples of the form {Key, Value} in List will be converted to an association of the form Key => Value. Anything else will be silently ignored.

If the same key appears in List multiple times, the value of the one appearing nearest to the head of List will be in the result map, that is the value that would be returned by a call to get_value(Key, List).

Example:

to_map([a, {b, 1}, {c, 2}, {c, 3}])

returns:

#{a => true, b => 1, c => 2}

to_map(List, Stages)

(since OTP 24.0)

-spec to_map(List, Stages) -> Map
                when
                    List :: [term()],
                    Stages :: [Operation],
                    Operation :: {aliases, Aliases} | {negations, Negations} | {expand, Expansions},
                    Aliases :: [{Key, Key}],
                    Negations :: [{Key, Key}],
                    Expansions :: [{Property :: property(), Expansion :: [term()]}],
                    Map :: #{term() => term()}.

Converts the property list List to a map after applying the normalizations given in Stages.

See also normalize/2, to_map/1.

unfold(ListIn)

-spec unfold(ListIn) -> ListOut when ListIn :: [term()], ListOut :: [term()].

Unfolds all occurrences of atoms in ListIn to tuples {Atom, true}.

See also compact/1.

qlc

This module provides a query interface to Mnesia, ETS, Dets, and other data structures that provide an iterator style traversal of objects.

Overview

This module provides a query interface to QLC tables. Typical QLC tables are Mnesia, ETS, and Dets tables. Support is also provided for user-defined tables, see section Implementing a QLC Table. A query is expressed using Query List Comprehensions (QLCs). The answers to a query are determined by data in QLC tables that fulfill the constraints expressed by the QLCs of the query. QLCs are similar to ordinary list comprehensions as described in Erlang Reference Manual and Programming Examples, except that variables introduced in patterns cannot be used in list expressions. In the absence of optimizations and options such as cache and unique (see section Common Options, every QLC free of QLC tables evaluates to the same list of answers as the identical ordinary list comprehension.

While ordinary list comprehensions evaluate to lists, calling q/1,2 returns a query handle. To obtain all the answers to a query, eval/1,2 is to be called with the query handle as first argument. Query handles are essentially functional objects (funs) created in the module calling q/1,2. As the funs refer to the module code, be careful not to keep query handles too long if the module code is to be replaced. Code replacement is described in section Compilation and Code Loading in the Erlang Reference Manual. The list of answers can also be traversed in chunks by use of a query cursor. Query cursors are created by calling cursor/1,2 with a query handle as first argument. Query cursors are essentially Erlang processes. One answer at a time is sent from the query cursor process to the process that created the cursor.

Syntax

Syntactically QLCs have the same parts as ordinary list comprehensions:

[Expression || Qualifier1, Qualifier2, ...]

Expression (the template) is any Erlang expression. Qualifiers are either filters or generators. Filters are Erlang expressions returning boolean/0. Generators have the form Pattern <- ListExpression, where ListExpression is an expression evaluating to a query handle or a list. Query handles are returned from append/1,2, keysort/2,3, q/1,2, sort/1,2, string_to_handle/1,2,3, and table/2.

Evaluation

A query handle is evaluated in the following order:

Inspection of options and the collection of information about tables. As a result, qualifiers are modified during the optimization phase.

All list expressions are evaluated. If a cursor has been created, evaluation takes place in the cursor process. For list expressions that are QLCs, the list expressions of the generators of the QLCs are evaluated as well. Be careful if list expressions have side effects, as list expressions are evaluated in unspecified order.

The answers are found by evaluating the qualifiers from left to right, backtracking when some filter returns false, or collecting the template when all filters return true.

Filters that do not return boolean/0 but fail are handled differently depending on their syntax: if the filter is a guard, it returns false, otherwise the query evaluation fails. This behavior makes it possible for the qlc module to do some optimizations without affecting the meaning of a query. For example, when testing some position of a table and one or more constants for equality, only the objects with equal values are candidates for further evaluation. The other objects are guaranteed to make the filter return false, but never fail. The (small) set of candidate objects can often be found by looking up some key values of the table or by traversing the table using a match specification. It is necessary to place the guard filters immediately after the table generator, otherwise the candidate objects are not restricted to a small set. The reason is that objects that could make the query evaluation fail must not be excluded by looking up a key or running a match specification.

Join

The qlc module supports fast join of two query handles. Fast join is possible if some position P1 of one query handler and some position P2 of another query handler are tested for equality. Two fast join methods are provided:

Lookup join traverses all objects of one query handle and finds objects of the other handle (a QLC table) such that the values at P1 and P2 match or compare equal. The qlc module does not create any indexes but looks up values using the key position and the indexed positions of the QLC table.

Merge join sorts the objects of each query handle if necessary and filters out objects where the values at P1 and P2 do not compare equal. If many objects with the same value of P2 exist, a temporary file is used for the equivalence classes.

The qlc module warns at compile time if a QLC combines query handles in such a way that more than one join is possible. That is, no query planner is provided that can select a good order between possible join operations. It is up to the user to order the joins by introducing query handles.

The join is to be expressed as a guard filter. The filter must be placed immediately after the two joined generators, possibly after guard filters that use variables from no other generators but the two joined generators. The qlc module inspects the operands of =:=/2, ==/2, is_record/2, element/2, and logical operators (and/2, or/2, andalso/2, orelse/2, xor/2) when determining which joins to consider.

Common Options

The following options are accepted by cursor/2, eval/2, fold/4, and info/2:

{cache_all, Cache}, where Cache is equal to ets or list adds a {cache, Cache} option to every list expression of the query except tables and lists. Defaults to {cache_all, no}. Option cache_all is equivalent to {cache_all, ets}.

{max_list_size, MaxListSize}, where MaxListSize is the size in bytes of terms on the external format. If the accumulated size of collected objects exceeds MaxListSize, the objects are written onto a temporary file. This option is used by option {cache, list} and by the merge join method. Defaults to 512*1024 bytes.

{tmpdir_usage, TmpFileUsage} determines the action taken when qlc is about to create temporary files on the directory set by option tmpdir. If the value is not_allowed, an error tuple is returned, otherwise temporary files are created as needed. Default is allowed, which means that no further action is taken. The values info_msg, warning_msg, and error_msg mean that the function with the corresponding name in module error_logger is called for printing some information (currently the stacktrace).

{tmpdir, TempDirectory} sets the directory used by merge join for temporary files and by option {cache, list}. The option also overrides option tmpdir of keysort/3 and sort/2. Defaults to "", which means that the directory returned by file:get_cwd() is used.

{unique_all, true} adds a {unique, true} option to every list expression of the query. Defaults to {unique_all, false}. Option unique_all is equivalent to {unique_all, true}.

Getting Started

As mentioned earlier, queries are expressed in the list comprehension syntax as described in section Expressions in Erlang Reference Manual. In the following, some familiarity with list comprehensions is assumed. The examples in section List Comprehensions in Programming Examples can get you started. Notice that list comprehensions do not add any computational power to the language; anything that can be done with list comprehensions can also be done without them. But they add syntax for expressing simple search problems, which is compact and clear once you get used to it.

Many list comprehension expressions can be evaluated by the qlc module. Exceptions are expressions, such that variables introduced in patterns (or filters) are used in some generator later in the list comprehension. As an example, consider an implementation of lists:append(L): [X ||Y <- L, X <- Y]. Y is introduced in the first generator and used in the second. The ordinary list comprehension is normally to be preferred when there is a choice as to which to use. One difference is that eval/1,2 collects answers in a list that is finally reversed, while list comprehensions collect answers on the stack that is finally unwound.

What the qlc module primarily adds to list comprehensions is that data can be read from QLC tables in small chunks. A QLC table is created by calling qlc:table/2. Usually qlc:table/2 is not called directly from the query but through an interface function of some data structure. Erlang/OTP includes a few examples of such functions: mnesia:table/1,2, ets:table/1,2, and dets:table/1,2. For a given data structure, many functions can create QLC tables, but common for these functions is that they return a query handle created by qlc:table/2. Using the QLC tables provided by Erlang/OTP is usually probably sufficient, but for the more advanced user section Implementing a QLC Table describes the implementation of a function calling qlc:table/2.

Besides qlc:table/2, other functions return query handles. They are used more seldom than tables, but are sometimes useful. qlc:append/1,2 traverses objects from many tables or lists after each other. If, for example, you want to traverse all answers to a query QH and then finish off by a term {finished}, you can do that by calling qlc:append(QH, [{finished}]). append/2 first returns all objects of QH, then {finished}. If a tuple {finished} exists among the answers to QH, it is returned twice from append/2.

As another example, consider concatenating the answers to two queries QH1 and QH2 while removing all duplicates. This is accomplished by using option unique:

qlc:q([X || X <- qlc:append(QH1, QH2)], {unique, true})

The cost is substantial: every returned answer is stored in an ETS table. Before returning an answer, it is looked up in the ETS table to check if it has already been returned. Without the unique option, all answers to QH1 would be returned followed by all answers to QH2. The unique option keeps the order between the remaining answers.

If the order of the answers is not important, there is an alternative to the unique option, namely to sort the answers uniquely:

qlc:sort(qlc:q([X || X <- qlc:append(QH1, QH2)], {unique, true})).

This query also removes duplicates but the answers are sorted. If there are many answers, temporary files are used. Notice that to get the first unique answer, all answers must be found and sorted. Both alternatives find duplicates by comparing answers, that is, if A1 and A2 are answers found in that order, then A2 is a removed if A1 == A2.

To return only a few answers, cursors can be used. The following code returns no more than five answers using an ETS table for storing the unique answers:

C = qlc:cursor(qlc:q([X || X <- qlc:append(QH1, QH2)],{unique,true})),
R = qlc:next_answers(C, 5),
ok = qlc:delete_cursor(C),
R.

QLCs are convenient for stating constraints on data from two or more tables. The following example does a natural join on two query handles on position 2:

qlc:q([{X1,X2,X3,Y1} ||
          {X1,X2,X3} <- QH1,
          {Y1,Y2} <- QH2,
          X2 =:= Y2])

The qlc module evaluates this differently depending on the query handles QH1 and QH2. If, for example, X2 is matched against the key of a QLC table, the lookup join method traverses the objects of QH2 while looking up key values in the table. However, if not X2 or Y2 is matched against the key or an indexed position of a QLC table, the merge join method ensures that QH1 and QH2 are both sorted on position 2 and next do the join by traversing the objects one by one.

Option join can be used to force the qlc module to use a certain join method. For the rest of this section it is assumed that the excessively slow join method called "nested loop" has been chosen:

qlc:q([{X1,X2,X3,Y1} ||
          {X1,X2,X3} <- QH1,
          {Y1,Y2} <- QH2,
          X2 =:= Y2],
      {join, nested_loop})

In this case the filter is applied to every possible pair of answers to QH1 and QH2, one at a time. If there are M answers to QH1 and N answers to QH2, the filter is run M*N times.

If QH2 is a call to the function for gb_trees, as defined in section Implementing a QLC Table, then gb_table:table/1 , the iterator for the gb-tree is initiated for each answer to QH1. The objects of the gb-tree are then returned one by one. This is probably the most efficient way of traversing the table in that case, as it takes minimal computational power to get the following object. But if QH2 is not a table but a more complicated QLC, it can be more efficient to use some RAM memory for collecting the answers in a cache, particularly if there are only a few answers. It must then be assumed that evaluating QH2 has no side effects so that the meaning of the query does not change if QH2 is evaluated only once. One way of caching the answers is to evaluate QH2 first of all and substitute the list of answers for QH2 in the query. Another way is to use option cache. It is expressed like this:

QH2' = qlc:q([X || X <- QH2], {cache, ets})

or only

QH2' = qlc:q([X || X <- QH2], cache)

The effect of option cache is that when generator QH2' is run the first time, every answer is stored in an ETS table. When the next answer of QH1 is tried, answers to QH2' are copied from the ETS table, which is very fast. As for option unique the cost is a possibly substantial amount of RAM memory.

Option {cache, list} offers the possibility to store the answers in a list on the process heap. This has the potential of being faster than ETS tables, as there is no need to copy answers from the table. However, it can often result in slower evaluation because of more garbage collections of the process heap and increased RAM memory consumption because of larger heaps. Another drawback with cache lists is that if the list size exceeds a limit, a temporary file is used. Reading the answers from a file is much slower than copying them from an ETS table. But if the available RAM memory is scarce, setting the limit to some low value is an alternative.

Option cache_all can be set to ets or list when evaluating a query. It adds a cache or {cache, list} option to every list expression except QLC tables and lists on all levels of the query. This can be used for testing if caching would improve efficiency at all. If the answer is yes, further testing is needed to pinpoint the generators that are to be cached.

Implementing a QLC Table

As an example of how to use function table/2, the implementation of a QLC table for the gb_trees module is given:

-module(gb_table).

-export([table/1]).

table(T) ->
    TF = fun() -> qlc_next(gb_trees:next(gb_trees:iterator(T))) end,
    InfoFun = fun(num_of_objects) -> gb_trees:size(T);
                 (keypos) -> 1;
                 (is_sorted_key) -> true;
                 (is_unique_objects) -> true;
                 (_) -> undefined
              end,
    LookupFun =
        fun(1, Ks) ->
                lists:flatmap(fun(K) ->
                                      case gb_trees:lookup(K, T) of
                                          {value, V} -> [{K,V}];
                                          none -> []
                                      end
                              end, Ks)
        end,
    FormatFun =
        fun({all, NElements, ElementFun}) ->
                ValsS = io_lib:format("gb_trees:from_orddict(~w)",
                                      [gb_nodes(T, NElements, ElementFun)]),
                io_lib:format("gb_table:table(~s)", [ValsS]);
           ({lookup, 1, KeyValues, _NElements, ElementFun}) ->
                ValsS = io_lib:format("gb_trees:from_orddict(~w)",
                                      [gb_nodes(T, infinity, ElementFun)]),
                io_lib:format("lists:flatmap(fun(K) -> "
                              "case gb_trees:lookup(K, ~s) of "
                              "{value, V} -> [{K,V}];none -> [] end "
                              "end, ~w)",
                              [ValsS, [ElementFun(KV) || KV <- KeyValues]])
        end,
    qlc:table(TF, [{info_fun, InfoFun}, {format_fun, FormatFun},
                   {lookup_fun, LookupFun},{key_equality,'=='}]).

qlc_next({X, V, S}) ->
    [{X,V} | fun() -> qlc_next(gb_trees:next(S)) end];
qlc_next(none) ->
    [].

gb_nodes(T, infinity, ElementFun) ->
    gb_nodes(T, -1, ElementFun);
gb_nodes(T, NElements, ElementFun) ->
    gb_iter(gb_trees:iterator(T), NElements, ElementFun).

gb_iter(_I, 0, _EFun) ->
    '...';
gb_iter(I0, N, EFun) ->
    case gb_trees:next(I0) of
        {X, V, I} ->
            [EFun({X,V}) | gb_iter(I, N-1, EFun)];
        none ->
            []
    end.

TF is the traversal function. The qlc module requires that there is a way of traversing all objects of the data structure. gb_trees has an iterator function suitable for that purpose. Notice that for each object returned, a new fun is created. As long as the list is not terminated by [], it is assumed that the tail of the list is a nullary function and that calling the function returns further objects (and functions).

The lookup function is optional. It is assumed that the lookup function always finds values much faster than it would take to traverse the table. The first argument is the position of the key. As qlc_next/1 returns the objects as {Key, Value} pairs, the position is 1. Notice that the lookup function is to return {Key, Value} pairs, as the traversal function does.

The format function is also optional. It is called by info/1,2 to give feedback at runtime of how the query is to be evaluated. Try to give as good feedback as possible without showing too much details. In the example, at most seven objects of the table are shown. The format function handles two cases: all means that all objects of the table are traversed; {lookup, 1, KeyValues} means that the lookup function is used for looking up key values.

Whether the whole table is traversed or only some keys looked up depends on how the query is expressed. If the query has the form

qlc:q([T || P <- LE, F])

and P is a tuple, the qlc module analyzes P and F in compile time to find positions of tuple P that are tested for equality to constants. If such a position at runtime turns out to be the key position, the lookup function can be used, otherwise all objects of the table must be traversed. The info function InfoFun returns the key position. There can be indexed positions as well, also returned by the info function. An index is an extra table that makes lookup on some position fast. Mnesia maintains indexes upon request, and introduces so called secondary keys. The qlc module prefers to look up objects using the key before secondary keys regardless of the number of constants to look up.

Key Equality

Erlang/OTP has two operators for testing term equality: ==/2 and =:=/2. The difference is all about the integers that can be represented by floats. For example, 2 == 2.0 evaluates to true while 2 =:= 2.0 evaluates to false. Normally this is a minor issue, but the qlc module cannot ignore the difference, which affects the user's choice of operators in QLCs.

If the qlc module at compile time can determine that some constant is free of integers, it does not matter which one of ==/2 or =:=/2 is used:

1> E1 = ets:new(t, [set]), % uses =:=/2 for key equality
Q1 = qlc:q([K ||
{K} <- ets:table(E1),
K == 2.71 orelse K == a]),
io:format("~s~n", [qlc:info(Q1)]).
ets:match_spec_run(
       lists:flatmap(fun(V) ->
			    ets:lookup(#Ref<0.3098908599.2283929601.256025>,
				       V)
		     end,
		     [a, 2.71]),
       ets:match_spec_compile([{{'$1'}, [], ['$1']}]))

In the example, operator ==/2 has been handled exactly as =:=/2 would have been handled. However, if it cannot be determined at compile time that some constant is free of integers, and the table uses =:=/2 when comparing keys for equality (see option key_equality), then the qlc module does not try to look up the constant. The reason is that there is in the general case no upper limit on the number of key values that can compare equal to such a constant; every combination of integers and floats must be looked up:

2> E2 = ets:new(t, [set]),
true = ets:insert(E2, [{{2,2},a},{{2,2.0},b},{{2.0,2},c}]),
F2 = fun(I) ->
qlc:q([V || {K,V} <- ets:table(E2), K == I])
end,
Q2 = F2({2,2}),
io:format("~s~n", [qlc:info(Q2)]).
ets:table(#Ref<0.3098908599.2283929601.256125>,
          [{traverse,
            {select,
             [{{'$1', '$2'}, [{'==', '$1', {const, {2, 2}}}], ['$2']}]}}])
3> lists:sort(qlc:e(Q2)).
[a,b,c]

Looking up only {2,2} would not return b and c.

If the table uses ==/2 when comparing keys for equality, the qlc module looks up the constant regardless of which operator is used in the QLC. However, ==/2 is to be preferred:

4> E3 = ets:new(t, [ordered_set]), % uses ==/2 for key equality
true = ets:insert(E3, [{{2,2.0},b}]),
F3 = fun(I) ->
qlc:q([V || {K,V} <- ets:table(E3), K == I])
end,
Q3 = F3({2,2}),
io:format("~s~n", [qlc:info(Q3)]).
ets:match_spec_run(ets:lookup(#Ref<0.3098908599.2283929601.256211>,
                              {2, 2}),
                   ets:match_spec_compile([{{'$1', '$2'}, [], ['$2']}]))
5> qlc:e(Q3).
[b]

Lookup join is handled analogously to lookup of constants in a table: if the join operator is ==/2, and the table where constants are to be looked up uses =:=/2 when testing keys for equality, then the qlc module does not consider lookup join for that table.

See Also

dets, erl_eval, erlang, error_logger, ets, file, file_sorter, mnesia, shell, Erlang Reference Manual, Programming Examples

Summary

Types

abstract_expr()

Parse trees for Erlang expression, see section The Abstract Format in the ERTS User's Guide.

answer()

answers()

cache()

key_pos()

match_expression()

Match specification, see section Match Specifications in Erlang in the ERTS User's Guide and ms_transform.

max_list_size()

no_files()

An integer > 1.

order()

order_fun()

query_cursor()

A query cursor.

query_handle()

A query handle.

query_handle_or_list()

query_list_comprehension()

A literal query list comprehension.

sort_option()

See file_sorter for a description of the options.

sort_options()

spawn_options()

tmp_directory()

tmp_file_usage()

Functions

append(QHL)

Returns a query handle. When evaluating query handle QH, all answers to the first query handle in QHL are returned, followed by all answers to the remaining query handles in QHL.

append(QH1, QH2)

Returns a query handle. When evaluating query handle QH3, all answers to QH1 are returned, followed by all answers to QH2.

cursor(QH)

Equivalent to cursor(QH, []).

cursor(QH, Options)

Creates a query cursor and makes the calling process the owner of the cursor.

delete_cursor(QueryCursor)

Deletes a query cursor. Only the owner of the cursor can delete the cursor.

e(QH)

Equivalent to eval(QH, []).

e(QH, Options)

Equivalent to eval(QH, Options).

eval(QH)

Equivalent to eval(QH, []).

eval(QH, Options)

Evaluates a query handle in the calling process and collects all answers in a list.

fold(Function, Acc0, QH)

Equivalent to fold(Function, Acc0, QH, []).

fold(Function, Acc0, QH, Options)

Calls Function on successive answers to the query handle together with an extra argument AccIn.

format_error(Error)

Returns a descriptive string in English of an error tuple returned by some of the functions of the qlc module or the parse transform. This function is mainly used by the compiler invoking the parse transform.

info(QH)

Equivalent to info(QH, []).

info(QH, Options)

Returns information about a query handle. The information describes the simplifications and optimizations that are the results of preparing the query for evaluation. This function is probably mainly useful during debugging.

keysort(KeyPos, QH1)

Equivalent to keysort(KeyPos, QH1, []).

keysort(KeyPos, QH1, SortOptions)

Returns a query handle. When evaluating query handle QH2, the answers to query handle QH1 are sorted by file_sorter:keysort/4 according to the options.

next_answers(QueryCursor)

Equivalent to next_answers(C, 10).

next_answers(QueryCursor, NumberOfAnswers)

Returns some or all of the remaining answers to a query cursor. Only the owner of QueryCursor can retrieve answers.

q(QLC)

Equivalent to q(QLC, []).

q(QLC, Options)

Returns a query handle for a QLC. The QLC must be the first argument to this function, otherwise it is evaluated as an ordinary list comprehension. It is also necessary to add the following line to the source code

sort(QH1)

Equivalent to sort(QH, []).

sort(QH1, SortOptions)

Returns a query handle. When evaluating query handle QH2, the answers to query handle QH1 are sorted by file_sorter:sort/3 according to the options.

string_to_handle(QueryString)

Equivalent to string_to_handle(QueryString, []).

string_to_handle(QueryString, Options)

Equivalent to string_to_handle(QueryString, Options, erl_eval:new_bindings()).

string_to_handle(QueryString, Options, Bindings)

A string version of q/1,2. When the query handle is evaluated, the fun created by the parse transform is interpreted by erl_eval. The query string is to be one single QLC terminated by a period.

table(TraverseFun, Options)

Returns a query handle for a QLC table. In Erlang/OTP there is support for ETS, Dets, and Mnesia tables, but many other data structures can be turned into QLC tables. This is accomplished by letting function(s) in the module implementing the data structure create a query handle by calling qlc:table/2.

Types

abstract_expr()

(not exported)

-type abstract_expr() :: erl_parse:abstract_expr().

Parse trees for Erlang expression, see section The Abstract Format in the ERTS User's Guide.

answer()

(not exported)

-type answer() :: term().

answers()

(not exported)

-type answers() :: [answer()].

cache()

(not exported)

-type cache() :: ets | list | no.

key_pos()

(not exported)

-type key_pos() :: pos_integer() | [pos_integer()].

match_expression()

(not exported)

-type match_expression() :: ets:match_spec().

Match specification, see section Match Specifications in Erlang in the ERTS User's Guide and ms_transform.

max_list_size()

(not exported)

-type max_list_size() :: non_neg_integer().

no_files()

(not exported)

-type no_files() :: pos_integer().

An integer > 1.

order()

(not exported)

-type order() :: ascending | descending | order_fun().

order_fun()

(not exported)

-type order_fun() :: fun((term(), term()) -> boolean()).

query_cursor()

-opaque query_cursor()

A query cursor.

query_handle()

-opaque query_handle()

A query handle.

query_handle_or_list()

(not exported)

-type query_handle_or_list() :: query_handle() | list().

query_list_comprehension()

(not exported)

-type query_list_comprehension() :: term().

A literal query list comprehension.

sort_option()

(not exported)

-type sort_option() ::
          {compressed, boolean()} |
          {no_files, no_files()} |
          {order, order()} |
          {size, pos_integer()} |
          {tmpdir, tmp_directory()} |
          {unique, boolean()}.

See file_sorter for a description of the options.

sort_options()

(not exported)

-type sort_options() :: [sort_option()] | sort_option().

spawn_options()

(not exported)

-type spawn_options() :: default | [proc_lib:spawn_option()].

tmp_directory()

(not exported)

-type tmp_directory() :: [] | file:name().

tmp_file_usage()

(not exported)

-type tmp_file_usage() :: allowed | not_allowed | info_msg | warning_msg | error_msg.

Functions

append(QHL)

-spec append(QHL) -> QH when QHL :: [query_handle_or_list()], QH :: query_handle().

Returns a query handle. When evaluating query handle QH, all answers to the first query handle in QHL are returned, followed by all answers to the remaining query handles in QHL.

append(QH1, QH2)

-spec append(QH1, QH2) -> QH3
                when QH1 :: query_handle_or_list(), QH2 :: query_handle_or_list(), QH3 :: query_handle().

Returns a query handle. When evaluating query handle QH3, all answers to QH1 are returned, followed by all answers to QH2.

append(QH1, QH2) is equivalent to append([QH1, QH2]).

cursor(QH)

-spec cursor(QH) -> Cursor when QH :: query_handle_or_list(), Cursor :: query_cursor().

Equivalent to cursor(QH, []).

cursor(QH, Options)

-spec cursor(QH, Options) -> Cursor
                when
                    QH :: query_handle_or_list(),
                    Options :: [Option] | Option,
                    Option ::
                        {cache_all, cache()} |
                        cache_all |
                        {max_list_size, max_list_size()} |
                        {spawn_options, spawn_options()} |
                        {tmpdir_usage, tmp_file_usage()} |
                        {tmpdir, tmp_directory()} |
                        {unique_all, boolean()} |
                        unique_all,
                    Cursor :: query_cursor().

Creates a query cursor and makes the calling process the owner of the cursor.

The cursor is to be used as argument to next_answers/1,2 and (eventually) delete_cursor/1. Calls erlang:spawn_opt/2 to spawn and link to a process that evaluates the query handle. The value of option spawn_options is used as last argument when calling spawn_opt/2. Defaults to [link].

Example:

1> QH = qlc:q([{X,Y} || X <- [a,b], Y <- [1,2]]),
QC = qlc:cursor(QH),
qlc:next_answers(QC, 1).
[{a,1}]
2> qlc:next_answers(QC, 1).
[{a,2}]
3> qlc:next_answers(QC, all_remaining).
[{b,1},{b,2}]
4> qlc:delete_cursor(QC).
ok

delete_cursor(QueryCursor)

-spec delete_cursor(QueryCursor) -> ok when QueryCursor :: query_cursor().

Deletes a query cursor. Only the owner of the cursor can delete the cursor.

e(QH)

-spec e(QH) -> Answers | Error
           when
               QH :: query_handle_or_list(),
               Answers :: answers(),
               Error :: {error, module(), Reason},
               Reason :: file_sorter:reason().

Equivalent to eval(QH, []).

e(QH, Options)

-spec e(QH, Options) -> Answers | Error
           when
               QH :: query_handle_or_list(),
               Options :: [Option] | Option,
               Option ::
                   {cache_all, cache()} |
                   cache_all |
                   {max_list_size, max_list_size()} |
                   {tmpdir_usage, tmp_file_usage()} |
                   {tmpdir, tmp_directory()} |
                   {unique_all, boolean()} |
                   unique_all,
               Answers :: answers(),
               Error :: {error, module(), Reason},
               Reason :: file_sorter:reason().

Equivalent to eval(QH, Options).

eval(QH)

-spec eval(QH) -> Answers | Error
              when
                  QH :: query_handle_or_list(),
                  Answers :: answers(),
                  Error :: {error, module(), Reason},
                  Reason :: file_sorter:reason().

Equivalent to eval(QH, []).

eval(QH, Options)

-spec eval(QH, Options) -> Answers | Error
              when
                  QH :: query_handle_or_list(),
                  Answers :: answers(),
                  Options :: [Option] | Option,
                  Option ::
                      {cache_all, cache()} |
                      cache_all |
                      {max_list_size, max_list_size()} |
                      {tmpdir_usage, tmp_file_usage()} |
                      {tmpdir, tmp_directory()} |
                      {unique_all, boolean()} |
                      unique_all,
                  Error :: {error, module(), Reason},
                  Reason :: file_sorter:reason().

Evaluates a query handle in the calling process and collects all answers in a list.

Example:

1> QH = qlc:q([{X,Y} || X <- [a,b], Y <- [1,2]]),
qlc:eval(QH).
[{a,1},{a,2},{b,1},{b,2}]

fold(Function, Acc0, QH)

-spec fold(Function, Acc0, QH) -> Acc1 | Error
              when
                  QH :: query_handle_or_list(),
                  Function :: fun((answer(), AccIn) -> AccOut),
                  Acc0 :: term(),
                  Acc1 :: term(),
                  AccIn :: term(),
                  AccOut :: term(),
                  Error :: {error, module(), Reason},
                  Reason :: file_sorter:reason().

Equivalent to fold(Function, Acc0, QH, []).

fold(Function, Acc0, QH, Options)

-spec fold(Function, Acc0, QH, Options) -> Acc1 | Error
              when
                  QH :: query_handle_or_list(),
                  Function :: fun((answer(), AccIn) -> AccOut),
                  Acc0 :: term(),
                  Acc1 :: term(),
                  AccIn :: term(),
                  AccOut :: term(),
                  Options :: [Option] | Option,
                  Option ::
                      {cache_all, cache()} |
                      cache_all |
                      {max_list_size, max_list_size()} |
                      {tmpdir_usage, tmp_file_usage()} |
                      {tmpdir, tmp_directory()} |
                      {unique_all, boolean()} |
                      unique_all,
                  Error :: {error, module(), Reason},
                  Reason :: file_sorter:reason().

Calls Function on successive answers to the query handle together with an extra argument AccIn.

The query handle and the function are evaluated in the calling process. Function must return a new accumulator, which is passed to the next call. Acc0 is returned if there are no answers to the query handle.

Example:

1> QH = [1,2,3,4,5,6],
qlc:fold(fun(X, Sum) -> X + Sum end, 0, QH).
21

format_error(Error)

-spec format_error(Error) -> Chars when Error :: {error, module(), term()}, Chars :: io_lib:chars().

Returns a descriptive string in English of an error tuple returned by some of the functions of the qlc module or the parse transform. This function is mainly used by the compiler invoking the parse transform.

info(QH)

-spec info(QH) -> Info when QH :: query_handle_or_list(), Info :: abstract_expr() | string().

Equivalent to info(QH, []).

info(QH, Options)

-spec info(QH, Options) -> Info
              when
                  QH :: query_handle_or_list(),
                  Options :: [Option] | Option,
                  Option :: EvalOption | ReturnOption,
                  EvalOption ::
                      {cache_all, cache()} |
                      cache_all |
                      {max_list_size, max_list_size()} |
                      {tmpdir_usage, tmp_file_usage()} |
                      {tmpdir, tmp_directory()} |
                      {unique_all, boolean()} |
                      unique_all,
                  ReturnOption ::
                      {depth, Depth} | {flat, boolean()} | {format, Format} | {n_elements, NElements},
                  Depth :: infinity | non_neg_integer(),
                  Format :: abstract_code | string,
                  NElements :: infinity | pos_integer(),
                  Info :: abstract_expr() | string().

Returns information about a query handle. The information describes the simplifications and optimizations that are the results of preparing the query for evaluation. This function is probably mainly useful during debugging.

The information has the form of an Erlang expression where QLCs most likely occur. Depending on the format functions of mentioned QLC tables, it is not certain that the information is absolutely accurate.

Options:

The default is to return a sequence of QLCs in a block, but if option {flat, false} is specified, one single QLC is returned.

The default is to return a string, but if option {format, abstract_code} is specified, abstract code is returned instead. In the abstract code, port identifiers, references, and pids are represented by strings.

The default is to return all elements in lists, but if option {n_elements, NElements} is specified, only a limited number of elements are returned.

The default is to show all parts of objects and match specifications, but if option {depth, Depth} is specified, parts of terms below a certain depth are replaced by '...'.

Examples:

In the following example two simple QLCs are inserted only to hold option {unique, true}:

1> QH = qlc:q([{X,Y} || X <- [x,y], Y <- [a,b]]),
io:format("~s~n", [qlc:info(QH, unique_all)]).
begin
    V1 =
        qlc:q([
               SQV ||
                   SQV <- [x, y]
              ],
              [{unique, true}]),
    V2 =
        qlc:q([
               SQV ||
                   SQV <- [a, b]
              ],
              [{unique, true}]),
    qlc:q([
           {X,Y} ||
               X <- V1,
               Y <- V2
          ],
          [{unique, true}])
end

In the following example QLC V2 has been inserted to show the joined generators and the join method chosen. A convention is used for lookup join: the first generator (G2) is the one traversed, the second (G1) is the table where constants are looked up.

1> E1 = ets:new(e1, []),
E2 = ets:new(e2, []),
true = ets:insert(E1, [{1,a},{2,b}]),
true = ets:insert(E2, [{a,1},{b,2}]),
Q = qlc:q([{X,Z,W} ||
{X, Z} <- ets:table(E1),
{W, Y} <- ets:table(E2),
X =:= Y]),
io:format("~s~n", [qlc:info(Q)]).
begin
    V1 =
        qlc:q([
               P0 ||
                   P0 = {W, Y} <-
                       ets:table(#Ref<0.3098908599.2283929601.256549>)
              ]),
    V2 =
        qlc:q([
               [G1 | G2] ||
                   G2 <- V1,
                   G1 <-
                       ets:table(#Ref<0.3098908599.2283929601.256548>),
                   element(2, G1) =:= element(1, G2)
              ],
              [{join, lookup}]),
    qlc:q([
           {X, Z, W} ||
               [{X, Z} | {W, Y}] <- V2
          ])
end

keysort(KeyPos, QH1)

-spec keysort(KeyPos, QH1) -> QH2
                 when KeyPos :: key_pos(), QH1 :: query_handle_or_list(), QH2 :: query_handle().

Equivalent to keysort(KeyPos, QH1, []).

keysort(KeyPos, QH1, SortOptions)

-spec keysort(KeyPos, QH1, SortOptions) -> QH2
                 when
                     KeyPos :: key_pos(),
                     SortOptions :: sort_options(),
                     QH1 :: query_handle_or_list(),
                     QH2 :: query_handle().

Returns a query handle. When evaluating query handle QH2, the answers to query handle QH1 are sorted by file_sorter:keysort/4 according to the options.

The sorter uses temporary files only if QH1 does not evaluate to a list and the size of the binary representation of the answers exceeds Size bytes, where Size is the value of option size.

next_answers(QueryCursor)

-spec next_answers(QueryCursor) -> Answers | Error
                      when
                          QueryCursor :: query_cursor(),
                          Answers :: answers(),
                          Error :: {error, module(), Reason},
                          Reason :: file_sorter:reason().

Equivalent to next_answers(C, 10).

next_answers(QueryCursor, NumberOfAnswers)

-spec next_answers(QueryCursor, NumberOfAnswers) -> Answers | Error
                      when
                          QueryCursor :: query_cursor(),
                          Answers :: answers(),
                          NumberOfAnswers :: all_remaining | pos_integer(),
                          Error :: {error, module(), Reason},
                          Reason :: file_sorter:reason().

Returns some or all of the remaining answers to a query cursor. Only the owner of QueryCursor can retrieve answers.

Argument NumberOfAnswers determines the maximum number of answers returned. If less than the requested number of answers is returned, subsequent calls to next_answers return [].

q(QLC)

-spec q(QLC) -> QH when QLC :: query_list_comprehension(), QH :: query_handle().

Equivalent to q(QLC, []).

q(QLC, Options)

-spec q(QLC, Options) -> QH
           when
               QH :: query_handle(),
               Options :: [Option] | Option,
               Option ::
                   {max_lookup, MaxLookup} |
                   {cache, cache()} |
                   cache |
                   {join, Join} |
                   {lookup, Lookup} |
                   {unique, boolean()} |
                   unique,
               MaxLookup :: non_neg_integer() | infinity,
               Join :: any | lookup | merge | nested_loop,
               Lookup :: boolean() | any,
               QLC :: query_list_comprehension().

Returns a query handle for a QLC. The QLC must be the first argument to this function, otherwise it is evaluated as an ordinary list comprehension. It is also necessary to add the following line to the source code:

-include_lib("stdlib/include/qlc.hrl").

This causes a parse transform to substitute a fun for the QLC. The (compiled) fun is called when the query handle is evaluated.

When calling qlc:q/1,2 from the Erlang shell, the parse transform is automatically called. When this occurs, the fun substituted for the QLC is not compiled but is evaluated by erl_eval. This is also true when expressions are evaluated by file:eval/1,2 or in the debugger.

To be explicit, this does not work:

...
A = [X || {X} <- [{1},{2}]],
QH = qlc:q(A),
...

Variable A is bound to the evaluated value of the list comprehension ([1,2]). The compiler complains with an error message ("argument is not a query list comprehension"); the shell process stops with a badarg reason.

Options:

Option {cache, ets} can be used to cache the answers to a QLC. The answers are stored in one ETS table for each cached QLC. When a cached QLC is evaluated again, answers are fetched from the table without any further computations. Therefore, when all answers to a cached QLC have been found, the ETS tables used for caching answers to the qualifiers of the QLC can be emptied. Option cache is equivalent to {cache, ets}.

Option {cache, list} can be used to cache the answers to a QLC like {cache, ets}. The difference is that the answers are kept in a list (on the process heap). If the answers would occupy more than a certain amount of RAM memory, a temporary file is used for storing the answers. Option max_list_size sets the limit in bytes and the temporary file is put on the directory set by option tmpdir.

Option cache has no effect if it is known that the QLC is to be evaluated at most once. This is always true for the top-most QLC and also for the list expression of the first generator in a list of qualifiers. Notice that in the presence of side effects in filters or callback functions, the answers to QLCs can be affected by option cache.

Option {unique, true} can be used to remove duplicate answers to a QLC. The unique answers are stored in one ETS table for each QLC. The table is emptied every time it is known that there are no more answers to the QLC. Option unique is equivalent to {unique, true}. If option unique is combined with option {cache, ets}, two ETS tables are used, but the full answers are stored in one table only. If option unique is combined with option {cache, list}, the answers are sorted twice using keysort/3; once to remove duplicates and once to restore the order.

Options cache and unique apply not only to the QLC itself but also to the results of looking up constants, running match specifications, and joining handles.

Example:

In the following example the cached results of the merge join are traversed for each value of A. Notice that without option cache the join would have been carried out three times, once for each value of A.

1> Q = qlc:q([{A,X,Z,W} ||
A <- [a,b,c],
{X,Z} <- [{a,1},{b,4},{c,6}],
{W,Y} <- [{2,a},{3,b},{4,c}],
X =:= Y],
{cache, list}),
io:format("~s~n", [qlc:info(Q)]).
begin
    V1 =
        qlc:q([
               P0 ||
                   P0 = {X, Z} <-
                       qlc:keysort(1, [{a, 1}, {b, 4}, {c, 6}], [])
              ]),
    V2 =
        qlc:q([
               P0 ||
                   P0 = {W, Y} <-
                       qlc:keysort(2, [{2, a}, {3, b}, {4, c}], [])
              ]),
    V3 =
        qlc:q([
               [G1 | G2] ||
                   G1 <- V1,
                   G2 <- V2,
                   element(1, G1) == element(2, G2)
              ],
              [{join, merge}, {cache, list}]),
    qlc:q([
           {A, X, Z, W} ||
               A <- [a, b, c],
               [{X, Z} | {W, Y}] <- V3,
               X =:= Y
          ])
end

sort/1,2 and keysort/2,3 can also be used for caching answers and for removing duplicates. When sorting answers are cached in a list, possibly stored on a temporary file, and no ETS tables are used.

Sometimes (see table/2) traversal of tables can be done by looking up key values, which is assumed to be fast. Under certain (rare) circumstances there can be too many key values to look up. Option {max_lookup, MaxLookup} can then be used to limit the number of lookups: if more than MaxLookup lookups would be required, no lookups are done but the table is traversed instead. Defaults to infinity, which means that there is no limit on the number of keys to look up.

Example:

In the following example, using the gb_table module from section Implementing a QLC Table, there are six keys to look up: {1,a}, {1,b}, {1,c}, {2,a}, {2,b}, and {2,c}. The reason is that the two elements of key {X, Y} are compared separately.

1> T = gb_trees:empty(),
QH = qlc:q([X || {{X,Y},_} <- gb_table:table(T),
((X == 1) or (X == 2)) andalso
((Y == a) or (Y == b) or (Y == c))]),
io:format("~s~n", [qlc:info(QH)]).
ets:match_spec_run(
       lists:flatmap(fun(K) ->
                            case
                                gb_trees:lookup(K,
                                                gb_trees:from_orddict([]))
                            of
                                {value, V} ->
                                    [{K, V}];
                                none ->
                                    []
                            end
                     end,
                     [{1, a},
                      {1, b},
                      {1, c},
                      {2, a},
                      {2, b},
                      {2, c}]),
       ets:match_spec_compile([{{{'$1', '$2'}, '_'},
                                [],
                                ['$1']}]))

Options:

Option {lookup, true} can be used to ensure that the qlc module looks up constants in some QLC table. If there are more than one QLC table among the list expressions of the generators, constants must be looked up in at least one of the tables. The evaluation of the query fails if there are no constants to look up. This option is useful when it would be unacceptable to traverse all objects in some table. Setting option lookup to false ensures that no constants are looked up ({max_lookup, 0} has the same effect). Defaults to any, which means that constants are looked up whenever possible.

Option {join, Join} can be used to ensure that a certain join method is used:

{join, lookup} invokes the lookup join method.

{join, merge} invokes the merge join method.

{join, nested_loop} invokes the method of matching every pair of objects from two handles. This method is mostly very slow.

The evaluation of the query fails if the qlc module cannot carry out the chosen join method. Defaults to any, which means that some fast join method is used if possible.

sort(QH1)

-spec sort(QH1) -> QH2 when QH1 :: query_handle_or_list(), QH2 :: query_handle().

Equivalent to sort(QH, []).

sort(QH1, SortOptions)

-spec sort(QH1, SortOptions) -> QH2
              when SortOptions :: sort_options(), QH1 :: query_handle_or_list(), QH2 :: query_handle().

Returns a query handle. When evaluating query handle QH2, the answers to query handle QH1 are sorted by file_sorter:sort/3 according to the options.

The sorter uses temporary files only if QH1 does not evaluate to a list and the size of the binary representation of the answers exceeds Size bytes, where Size is the value of option size.

string_to_handle(QueryString)

-spec string_to_handle(QueryString) -> QH | Error
                          when
                              QueryString :: string(),
                              QH :: query_handle(),
                              Error :: {error, module(), Reason},
                              Reason :: erl_parse:error_info() | erl_scan:error_info().

Equivalent to string_to_handle(QueryString, []).

string_to_handle(QueryString, Options)

-spec string_to_handle(QueryString, Options) -> QH | Error
                          when
                              QueryString :: string(),
                              Options :: [Option] | Option,
                              Option ::
                                  {max_lookup, MaxLookup} |
                                  {cache, cache()} |
                                  cache |
                                  {join, Join} |
                                  {lookup, Lookup} |
                                  {unique, boolean()} |
                                  unique,
                              MaxLookup :: non_neg_integer() | infinity,
                              Join :: any | lookup | merge | nested_loop,
                              Lookup :: boolean() | any,
                              QH :: query_handle(),
                              Error :: {error, module(), Reason},
                              Reason :: erl_parse:error_info() | erl_scan:error_info().

Equivalent to string_to_handle(QueryString, Options, erl_eval:new_bindings()).

string_to_handle(QueryString, Options, Bindings)

-spec string_to_handle(QueryString, Options, Bindings) -> QH | Error
                          when
                              QueryString :: string(),
                              Options :: [Option] | Option,
                              Option ::
                                  {max_lookup, MaxLookup} |
                                  {cache, cache()} |
                                  cache |
                                  {join, Join} |
                                  {lookup, Lookup} |
                                  {unique, boolean()} |
                                  unique,
                              MaxLookup :: non_neg_integer() | infinity,
                              Join :: any | lookup | merge | nested_loop,
                              Lookup :: boolean() | any,
                              Bindings :: erl_eval:binding_struct(),
                              QH :: query_handle(),
                              Error :: {error, module(), Reason},
                              Reason :: erl_parse:error_info() | erl_scan:error_info().

A string version of q/1,2. When the query handle is evaluated, the fun created by the parse transform is interpreted by erl_eval. The query string is to be one single QLC terminated by a period.

Example:

1> L = [1,2,3],
Bs = erl_eval:add_binding('L', L, erl_eval:new_bindings()),
QH = qlc:string_to_handle("[X+1 || X <- L].", [], Bs),
qlc:eval(QH).
[2,3,4]

This function is probably mainly useful when called from outside of Erlang, for example from a driver written in C.

Note

Query handles created this way may have worse performance than when created directly via q/1,2.

table(TraverseFun, Options)

-spec table(TraverseFun, Options) -> QH
               when
                   TraverseFun :: TraverseFun0 | TraverseFun1,
                   TraverseFun0 :: fun(() -> TraverseResult),
                   TraverseFun1 :: fun((match_expression()) -> TraverseResult),
                   TraverseResult :: Objects | term(),
                   Objects :: [] | [term() | ObjectList],
                   ObjectList :: TraverseFun0 | Objects,
                   Options :: [Option] | Option,
                   Option ::
                       {format_fun, FormatFun} |
                       {info_fun, InfoFun} |
                       {lookup_fun, LookupFun} |
                       {parent_fun, ParentFun} |
                       {post_fun, PostFun} |
                       {pre_fun, PreFun} |
                       {key_equality, KeyComparison},
                   FormatFun :: undefined | fun((SelectedObjects) -> FormatedTable),
                   SelectedObjects ::
                       all |
                       {all, NElements, DepthFun} |
                       {match_spec, match_expression()} |
                       {lookup, Position, Keys} |
                       {lookup, Position, Keys, NElements, DepthFun},
                   NElements :: infinity | pos_integer(),
                   DepthFun :: fun((term()) -> term()),
                   FormatedTable :: {Mod, Fun, Args} | abstract_expr() | string(),
                   InfoFun :: undefined | fun((InfoTag) -> InfoValue),
                   InfoTag :: indices | is_unique_objects | keypos | num_of_objects,
                   InfoValue :: undefined | term(),
                   LookupFun :: undefined | fun((Position, Keys) -> LookupResult),
                   LookupResult :: [term()] | term(),
                   ParentFun :: undefined | fun(() -> ParentFunValue),
                   PostFun :: undefined | fun(() -> term()),
                   PreFun :: undefined | fun((PreArgs) -> term()),
                   PreArgs :: [PreArg],
                   PreArg :: {parent_value, ParentFunValue} | {stop_fun, StopFun},
                   ParentFunValue :: undefined | term(),
                   StopFun :: undefined | fun(() -> term()),
                   KeyComparison :: '=:=' | '==',
                   Position :: pos_integer(),
                   Keys :: [term()],
                   Mod :: atom(),
                   Fun :: atom(),
                   Args :: [term()],
                   QH :: query_handle().

Returns a query handle for a QLC table. In Erlang/OTP there is support for ETS, Dets, and Mnesia tables, but many other data structures can be turned into QLC tables. This is accomplished by letting function(s) in the module implementing the data structure create a query handle by calling qlc:table/2.

The different ways to traverse the table and properties of the table are handled by callback functions provided as options to qlc:table/2.

Callback function TraverseFun is used for traversing the table. It is to return a list of objects terminated by either [] or a nullary fun to be used for traversing the not yet traversed objects of the table. Any other return value is immediately returned as value of the query evaluation. Unary TraverseFuns are to accept a match specification as argument. The match specification is created by the parse transform by analyzing the pattern of the generator calling qlc:table/2 and filters using variables introduced in the pattern. If the parse transform cannot find a match specification equivalent to the pattern and filters, TraverseFun is called with a match specification returning every object.

Modules that can use match specifications for optimized traversal of tables are to call qlc:table/2 with an unary TraverseFun. An example is ets:table/2.

Other modules can provide a nullary TraverseFun. An example is gb_table:table/1 in section Implementing a QLC Table.

Unary callback function PreFun is called once before the table is read for the first time. If the call fails, the query evaluation fails.

Argument PreArgs is a list of tagged values. There are two tags, parent_value and stop_fun, used by Mnesia for managing transactions.

The value of parent_value is the value returned by ParentFun, or undefined if there is no ParentFun. ParentFun is called once just before the call of PreFun in the context of the process calling eval/1,2, fold/3,4, or cursor/1,2.

The value of stop_fun is a nullary fun that deletes the cursor if called from the parent, or undefined if there is no cursor.

Nullary callback function PostFun is called once after the table was last read. The return value, which is caught, is ignored. If PreFun has been called for a table, PostFun is guaranteed to be called for that table, even if the evaluation of the query fails for some reason.

The pre (post) functions for different tables are evaluated in unspecified order.

Other table access than reading, such as calling InfoFun, is assumed to be OK at any time.

Binary callback function LookupFun is used for looking up objects in the table. The first argument Position is the key position or an indexed position and the second argument Keys is a sorted list of unique values. The return value is to be a list of all objects (tuples), such that the element at Position is a member of Keys. Any other return value is immediately returned as value of the query evaluation. LookupFun is called instead of traversing the table if the parse transform at compile time can determine that the filters match and compare the element at Position in such a way that only Keys need to be looked up to find all potential answers.

The key position is obtained by calling InfoFun(keypos) and the indexed positions by calling InfoFun(indices). If the key position can be used for lookup, it is always chosen, otherwise the indexed position requiring the least number of lookups is chosen. If there is a tie between two indexed positions, the one occurring first in the list returned by InfoFun is chosen. Positions requiring more than max_lookup lookups are ignored.

Unary callback function InfoFun is to return information about the table. undefined is to be returned if the value of some tag is unknown:

indices - Returns a list of indexed positions, a list of positive integers.

is_unique_objects - Returns true if the objects returned by TraverseFun are unique.

keypos - Returns the position of the table key, a positive integer.

is_sorted_key - Returns true if the objects returned by TraverseFun are sorted on the key.

num_of_objects - Returns the number of objects in the table, a non-negative integer.

Unary callback function FormatFun is used by info/1,2 for displaying the call that created the query handle of the table. Defaults to undefined, which means that info/1,2 displays a call to '$MOD':'$FUN'/0. It is up to FormatFun to present the selected objects of the table in a suitable way. However, if a character list is chosen for presentation, it must be an Erlang expression that can be scanned and parsed (a trailing dot is added by info/1,2 though).

FormatFun is called with an argument that describes the selected objects based on optimizations done as a result of analyzing the filters of the QLC where the call to qlc:table/2 occurs. The argument can have the following values:

{lookup, Position, Keys, NElements, DepthFun}. - LookupFun is used for looking up objects in the table.

{match_spec, MatchExpression} - No way of finding all possible answers by looking up keys was found, but the filters could be transformed into a match specification. All answers are found by calling TraverseFun(MatchExpression).

{all, NElements, DepthFun} - No optimization was found. A match specification matching all objects is used if TraverseFun is unary.

NElements is the value of the info/1,2 option n_elements.

DepthFun is a function that can be used for limiting the size of terms; calling DepthFun(Term) substitutes '...' for parts of Term below the depth specified by the info/1,2 option depth.

If calling FormatFun with an argument including NElements and DepthFun fails, FormatFun is called once again with an argument excluding NElements and DepthFun ({lookup, Position, Keys} or all).

The value of option key_equality is to be '=:=' if the table considers two keys equal if they match, and to be '==' if two keys are equal if they compare equal. Defaults to '=:='.

For the various options recognized by table/1,2 in respective module, see ets, dets, and mnesia.

queue

Abstract data type for FIFO queues.

This module provides (double-ended) FIFO queues in an efficient manner.

All functions fail with reason badarg if arguments are of wrong type, for example, queue arguments are not queues, indexes are not integers, and list arguments are not lists. Improper lists cause internal crashes. An index out of range for a queue also causes a failure with reason badarg.

Some functions, where noted, fail with reason empty for an empty queue.

The data representing a queue as used by this module is to be regarded as opaque by other modules. In abstract terms, the representation is a composite type of existing Erlang terms. See note on data types. Any code assuming knowledge of the format is running on thin ice.

All operations have an amortized O(1) running time, except all/2, any/2, delete/2, delete_r/2, delete_with/2, delete_with_r/2, filter/2, filtermap/2, fold/3, join/2, len/1, member/2, split/2 that have O(n). To minimize the size of a queue minimizing the amount of garbage built by queue operations, the queues do not contain explicit length information, and that is why len/1 is O(n). If better performance for this particular operation is essential, it is easy for the caller to keep track of the length.

Queues are double-ended. The mental picture of a queue is a line of people (items) waiting for their turn. The queue front is the end with the item that has waited the longest. The queue rear is the end an item enters when it starts to wait. If instead using the mental picture of a list, the front is called head and the rear is called tail.

Entering at the front and exiting at the rear are reverse operations on the queue.

This module has three sets of interface functions: the "Original API", the "Extended API", and the "Okasaki API".

The "Original API" and the "Extended API" both use the mental picture of a waiting line of items. Both have reverse operations suffixed "_r".

The "Original API" item removal functions return compound terms with both the removed item and the resulting queue. The "Extended API" contains alternative functions that build less garbage and functions for just inspecting the queue ends. Also the "Okasaki API" functions build less garbage.

The "Okasaki API" is inspired by "Purely Functional Data Structures" by Chris Okasaki. It regards queues as lists. This API is by many regarded as strange and avoidable. For example, many reverse operations have lexically reversed names, some with more readable but perhaps less understandable aliases.

Summary

Types

queue()

queue(Item)

As returned by new/0.

Extended API

drop(Q1)

Returns a queue Q2 that is the result of removing the front item from Q1.

drop_r(Q1)

Returns a queue Q2 that is the result of removing the rear item from Q1.

get(Q)

Returns Item at the front of queue Q.

get_r(Q)

Returns Item at the rear of queue Q.

peek(Q)

Returns tuple {value, Item}, where Item is the front item of Q, or empty if Q is empty.

peek_r(Q)

Returns tuple {value, Item}, where Item is the rear item of Q, or empty if Q is empty.

Okasaki API

cons(Item, Q1)

Inserts Item at the head of queue Q1. Returns the new queue Q2.

daeh(Q)

Returns the tail item of queue Q.

head(Q)

Returns Item from the head of queue Q.

init(Q1)

Returns a queue Q2 that is the result of removing the tail item from Q1.

lait(Q1) deprecated

Returns a queue Q2 that is the result of removing the tail item from Q1.

last(Q)

Returns the tail item of queue Q.

liat(Q1)

Returns a queue Q2 that is the result of removing the tail item from Q1.

snoc(Q1, Item)

Inserts Item as the tail item of queue Q1. Returns the new queue Q2.

tail(Q1)

Returns a queue Q2 that is the result of removing the head item from Q1.

Original API

all(Pred, Q)

Returns true if Pred(Item) returns true for all items Item in Q, otherwise false.

any(Pred, Q)

Returns true if Pred(Item) returns true for at least one item Item in Q, otherwise false.

delete(Item, Q1)

Returns a copy of Q1 where the first item matching Item is deleted, if there is such an item.

delete_r(Item, Q1)

Returns a copy of Q1 where the last item matching Item is deleted, if there is such an item.

delete_with(Pred, Q1)

Returns a copy of Q1 where the first item for which Pred returns true is deleted, if there is such an item.

delete_with_r(Pred, Q1)

Returns a copy of Q1 where the last item for which Pred returns true is deleted, if there is such an item.

filter(Fun, Q1)

Returns a queue Q2 that is the result of calling Fun(Item) on all items in Q1.

filtermap(Fun, Q1)

Returns a queue Q2 that is the result of calling Fun(Item) on all items in Q1.

fold(Fun, Acc0, Q)

Calls Fun(Item, AccIn) on successive items Item of Queue, starting with AccIn == Acc0. The queue is traversed in queue order, that is, from front to rear. Fun/2 must return a new accumulator, which is passed to the next call. The function returns the final value of the accumulator. Acc0 is returned if the queue is empty.

from_list(L)

Returns a queue containing the items in L in the same order; the head item of the list becomes the front item of the queue.

in(Item, Q1)

Inserts Item at the rear of queue Q1. Returns the resulting queue Q2.

in_r(Item, Q1)

Inserts Item at the front of queue Q1. Returns the resulting queue Q2.

is_empty(Q)

Tests if Q is empty and returns true if so, otherwise false.

is_queue(Term)

Tests if Term is a queue and returns true if so, otherwise false. Note that the test will return true for a term coinciding with the representation of a queue, even when not constructed by thus module. See also note on data types.

join(Q1, Q2)

Returns a queue Q3 that is the result of joining Q1 and Q2 with Q1 in front of Q2.

len(Q)

Calculates and returns the length of queue Q.

member(Item, Q)

Returns true if Item matches some element in Q, otherwise false.

new()

Returns an empty queue.

out(Q1)

Removes the item at the front of queue Q1. Returns tuple {{value, Item}, Q2}, where Item is the item removed and Q2 is the resulting queue. If Q1 is empty, tuple {empty, Q1} is returned.

out_r(Q1)

Removes the item at the rear of queue Q1. Returns tuple {{value, Item}, Q2}, where Item is the item removed and Q2 is the new queue. If Q1 is empty, tuple {empty, Q1} is returned.

reverse(Q1)

Returns a queue Q2 containing the items of Q1 in the reverse order.

split(N, Q1)

Splits Q1 in two. The N front items are put in Q2 and the rest in Q3.

to_list(Q)

Returns a list of the items in the queue in the same order; the front item of the queue becomes the head of the list.

Types

queue()

-type queue() :: queue(_).

queue(Item)

-opaque queue(Item)

As returned by new/0.

Extended API

drop(Q1)

-spec drop(Q1 :: queue(Item)) -> Q2 :: queue(Item).

Returns a queue Q2 that is the result of removing the front item from Q1.

Fails with reason empty if Q1 is empty.

Example:

1> Queue = queue:from_list([1,2,3,4,5]).
{[5,4,3],[1,2]}
2> Queue = queue:drop(Queue).
{[5,4,3],[2]}
3> queue:to_list(Queue1).
[2,3,4,5]

drop_r(Q1)

-spec drop_r(Q1 :: queue(Item)) -> Q2 :: queue(Item).

Returns a queue Q2 that is the result of removing the rear item from Q1.

Fails with reason empty if Q1 is empty.

Example:

1> Queue = queue:from_list([1,2,3,4,5]).
{[5,4,3],[1,2]}
2> Queue = queue:drop_r(Queue).
{[4,3],[1,2]}
3> queue:to_list(Queue1).
[1,2,3,4]

get(Q)

-spec get(Q :: queue(Item)) -> Item.

Returns Item at the front of queue Q.

Fails with reason empty if Q is empty.

Example 1:

1> Queue = queue:from_list([1,2,3,4,5]).
{[5,4,3],[1,2]}
2> 1 == queue:get(Queue).
true

get_r(Q)

-spec get_r(Q :: queue(Item)) -> Item.

Returns Item at the rear of queue Q.

Fails with reason empty if Q is empty.

Example 1:

1> Queue = queue:from_list([1,2,3,4,5]).
{[5,4,3],[1,2]}
2> 5 == queue:get_r(Queue).
true

peek(Q)

-spec peek(Q :: queue(Item)) -> empty | {value, Item}.

Returns tuple {value, Item}, where Item is the front item of Q, or empty if Q is empty.

Example 1:

1> queue:peek(queue:new()).
empty
2> Queue = queue:from_list([1,2,3,4,5]).
{[5,4,3],[1,2]}
3> queue:peek(Queue).
{value, 1}

peek_r(Q)

-spec peek_r(Q :: queue(Item)) -> empty | {value, Item}.

Returns tuple {value, Item}, where Item is the rear item of Q, or empty if Q is empty.

Example 1:

1> queue:peek_r(queue:new()).
empty
2> Queue = queue:from_list([1,2,3,4,5]).
{[5,4,3],[1,2]}
3> queue:peek_r(Queue).
{value, 5}

Okasaki API

cons(Item, Q1)

-spec cons(Item, Q1 :: queue(Item)) -> Q2 :: queue(Item).

Inserts Item at the head of queue Q1. Returns the new queue Q2.

Example:

1> Queue = queue:cons(0, queue:from_list([1,2,3])).
{[3,2],[0,1]}
2> queue:to_list(Queue).
[0,1,2,3]

daeh(Q)

-spec daeh(Q :: queue(Item)) -> Item.

Returns the tail item of queue Q.

Fails with reason empty if Q is empty.

Example 1:

1> queue:daeh(queue:from_list([1,2,3])).
3

head(Q)

-spec head(Q :: queue(Item)) -> Item.

Returns Item from the head of queue Q.

Fails with reason empty if Q is empty.

Example 1:

1> queue:head(queue:from_list([1,2,3])).
1

init(Q1)

-spec init(Q1 :: queue(Item)) -> Q2 :: queue(Item).

Returns a queue Q2 that is the result of removing the tail item from Q1.

Fails with reason empty if Q1 is empty.

Example:

1> Queue = queue:init(queue:from_list([1,2,3])).
{[2],[1]}
2> queue:to_list(Queue).
[1,2]

lait(Q1)

This function is deprecated. queue:lait/1 is deprecated; use queue:liat/1 instead.

-spec lait(Q1 :: queue(Item)) -> Q2 :: queue(Item).

Returns a queue Q2 that is the result of removing the tail item from Q1.

Fails with reason empty if Q1 is empty.

The name lait/1 is a misspelling - do not use it anymore.

last(Q)

-spec last(Q :: queue(Item)) -> Item.

Returns the tail item of queue Q.

Fails with reason empty if Q is empty.

Example:

1> queue:last(queue:from_list([1,2,3])).
3

liat(Q1)

-spec liat(Q1 :: queue(Item)) -> Q2 :: queue(Item).

Returns a queue Q2 that is the result of removing the tail item from Q1.

Fails with reason empty if Q1 is empty.

Example:

1> Queue = queue:liat(queue:from_list([1,2,3])).
{[2],[1]}
2> queue:to_list(Queue).
[1,2]

snoc(Q1, Item)

-spec snoc(Q1 :: queue(Item), Item) -> Q2 :: queue(Item).

Inserts Item as the tail item of queue Q1. Returns the new queue Q2.

Example:

1> Queue = queue:snoc(queue:from_list([1,2,3]), 4).
{[4,3,2],[1]}
2> queue:to_list(Queue).
[1,2,3,4]

tail(Q1)

-spec tail(Q1 :: queue(Item)) -> Q2 :: queue(Item).

Returns a queue Q2 that is the result of removing the head item from Q1.

Fails with reason empty if Q1 is empty.

Original API

all(Pred, Q)

(since OTP 24.0)

-spec all(Pred, Q :: queue(Item)) -> boolean() when Pred :: fun((Item) -> boolean()).

Returns true if Pred(Item) returns true for all items Item in Q, otherwise false.

Example:

1> Queue = queue:from_list([1,2,3,4,5]).
2> queue:all(fun (E) -> E > 3 end, Queue).
false
3> queue:all(fun (E) -> E > 0 end, Queue).
true

any(Pred, Q)

(since OTP 24.0)

-spec any(Pred, Q :: queue(Item)) -> boolean() when Pred :: fun((Item) -> boolean()).

Returns true if Pred(Item) returns true for at least one item Item in Q, otherwise false.

Example:

1> Queue = queue:from_list([1,2,3,4,5]).
2> queue:any(fun (E) -> E > 10 end, Queue).
false
3> queue:any(fun (E) -> E > 3 end, Queue).
true

delete(Item, Q1)

(since OTP 24.0)

-spec delete(Item, Q1) -> Q2 when Item :: T, Q1 :: queue(T), Q2 :: queue(T), T :: term().

Returns a copy of Q1 where the first item matching Item is deleted, if there is such an item.

Example:

1> Queue = queue:from_list([1,2,3,4,5]).
2> Queue1 = queue:delete(3, Queue).
3> queue:member(3, Queue1).
false

delete_r(Item, Q1)

(since OTP 24.0)

-spec delete_r(Item, Q1) -> Q2 when Item :: T, Q1 :: queue(T), Q2 :: queue(T), T :: term().

Returns a copy of Q1 where the last item matching Item is deleted, if there is such an item.

Example:

1> Queue = queue:from_list([1,2,3,4,3,5]).
2> Queue1 = queue:delete_r(3, Queue).
3> queue:to_list(Queue1).
[1,2,3,4,5]

delete_with(Pred, Q1)

(since OTP 24.0)

-spec delete_with(Pred, Q1) -> Q2
                     when
                         Pred :: fun((Item) -> boolean()),
                         Q1 :: queue(Item),
                         Q2 :: queue(Item),
                         Item :: term().

Returns a copy of Q1 where the first item for which Pred returns true is deleted, if there is such an item.

Example:

1> Queue = queue:from_list([100,1,2,3,4,5]).
2> Queue1 = queue:delete_with(fun (E) -> E > 0, Queue).
3> queue:to_list(Queue1).
[1,2,3,4,5]

delete_with_r(Pred, Q1)

(since OTP 24.0)

-spec delete_with_r(Pred, Q1) -> Q2
                       when
                           Pred :: fun((Item) -> boolean()),
                           Q1 :: queue(Item),
                           Q2 :: queue(Item),
                           Item :: term().

Returns a copy of Q1 where the last item for which Pred returns true is deleted, if there is such an item.

Example:

1> Queue = queue:from_list([1,2,3,4,5,100]).
2> Queue1 = queue:delete_with(fun (E) -> E > 10, Queue).
3> queue:to_list(Queue1).
[1,2,3,4,5]

filter(Fun, Q1)

-spec filter(Fun, Q1 :: queue(Item)) -> Q2 :: queue(Item) when Fun :: fun((Item) -> boolean() | [Item]).

Returns a queue Q2 that is the result of calling Fun(Item) on all items in Q1.

If Fun(Item) returns true, Item is copied to the result queue. If it returns false, Item is not copied. If it returns a list, the list elements are inserted instead of Item in the result queue.

Example 1:

1> Queue = queue:from_list([1,2,3,4,5]).
{[5,4,3],[1,2]}
2> Queue1 = queue:filter(fun (E) -> E > 2 end, Queue).
{[5],[3,4]}
3> queue:to_list(Queue1).
[3,4,5]

So, Fun(Item) returning [Item] is thereby semantically equivalent to returning true, just as returning [] is semantically equivalent to returning false. But returning a list builds more garbage than returning an atom.

Example 2:

1> Queue = queue:from_list([1,2,3,4,5]).
{[5,4,3],[1,2]}
2> Queue1 = queue:filter(fun (E) -> [E, E+1] end, Queue).
{[6,5,5,4,4,3],[1,2,2,3]}
3> queue:to_list(Queue1).
[1,2,2,3,3,4,4,5,5,6]

filtermap(Fun, Q1)

(since OTP 24.0)

-spec filtermap(Fun, Q1) -> Q2
                   when
                       Fun :: fun((Item) -> boolean() | {true, Value}),
                       Q1 :: queue(Item),
                       Q2 :: queue(Item | Value),
                       Item :: term(),
                       Value :: term().

Returns a queue Q2 that is the result of calling Fun(Item) on all items in Q1.

If Fun(Item) returns true, Item is copied to the result queue. If it returns false, Item is not copied. If it returns {true, NewItem}, the queue element at this position is replaced with NewItem in the result queue.

Example 1:

1> Queue = queue:from_list([1,2,3,4,5]).
{[5,4,3],[1,2]}
2> Queue1 = queue:filtermap(fun (E) -> E > 2 end, Queue).
{[5],[3,4]}
3> queue:to_list(Queue1).
[3,4,5]
4> Queue1 = queue:filtermap(fun (E) -> {true, E+100} end, Queue).
{"ihg","ef"}
5> queue:to_list(Queue1).
"efghi

fold(Fun, Acc0, Q)

(since OTP 24.0)

-spec fold(Fun, Acc0, Q :: queue(Item)) -> Acc1
              when
                  Fun :: fun((Item, AccIn) -> AccOut),
                  Acc0 :: term(),
                  Acc1 :: term(),
                  AccIn :: term(),
                  AccOut :: term().

Calls Fun(Item, AccIn) on successive items Item of Queue, starting with AccIn == Acc0. The queue is traversed in queue order, that is, from front to rear. Fun/2 must return a new accumulator, which is passed to the next call. The function returns the final value of the accumulator. Acc0 is returned if the queue is empty.

Example:

1> queue:fold(fun(X, Sum) -> X + Sum end, 0, queue:from_list([1,2,3,4,5])).
15
2> queue:fold(fun(X, Prod) -> X * Prod end, 1, queue:from_list([1,2,3,4,5])).
120

from_list(L)

-spec from_list(L :: [Item]) -> queue(Item).

Returns a queue containing the items in L in the same order; the head item of the list becomes the front item of the queue.

in(Item, Q1)

-spec in(Item, Q1 :: queue(Item)) -> Q2 :: queue(Item).

Inserts Item at the rear of queue Q1. Returns the resulting queue Q2.

Example:

1> Queue = queue:from_list([1,2,3,4,5]).
{[5,4,3],[1,2]}
2> Queue1 = queue:in(100, Queue).
{[100,5,4,3],[1,2]}
3> queue:to_list(Queue1).
[1,2,3,4,5,100]

in_r(Item, Q1)

-spec in_r(Item, Q1 :: queue(Item)) -> Q2 :: queue(Item).

Inserts Item at the front of queue Q1. Returns the resulting queue Q2.

Example:

1> Queue = queue:from_list([1,2,3,4,5]).
{[5,4,3],[1,2]}
2> Queue1 = queue:in_r(100, Queue).
{[5,4,3],[100,1,2]}
3> queue:to_list(Queue1).
[100,1,2,3,4,5]

is_empty(Q)

-spec is_empty(Q :: queue()) -> boolean().

Tests if Q is empty and returns true if so, otherwise false.

is_queue(Term)

-spec is_queue(Term :: term()) -> boolean().

Tests if Term is a queue and returns true if so, otherwise false. Note that the test will return true for a term coinciding with the representation of a queue, even when not constructed by thus module. See also note on data types.

join(Q1, Q2)

-spec join(Q1 :: queue(Item), Q2 :: queue(Item)) -> Q3 :: queue(Item).

Returns a queue Q3 that is the result of joining Q1 and Q2 with Q1 in front of Q2.

Example:

1> Queue1 = queue:from_list([1,3]).
{[3],[1]}
2> Queue2 = queue:from_list([2,4]).
{[4],[2]}
3> queue:to_list(queue:join(Queue1, Queue2)).
[1,3,2,4]

len(Q)

-spec len(Q :: queue()) -> non_neg_integer().

Calculates and returns the length of queue Q.

member(Item, Q)

-spec member(Item, Q :: queue(Item)) -> boolean().

Returns true if Item matches some element in Q, otherwise false.

new()

-spec new() -> queue(none()).

Returns an empty queue.

out(Q1)

-spec out(Q1 :: queue(Item)) -> {{value, Item}, Q2 :: queue(Item)} | {empty, Q1 :: queue(Item)}.

Removes the item at the front of queue Q1. Returns tuple {{value, Item}, Q2}, where Item is the item removed and Q2 is the resulting queue. If Q1 is empty, tuple {empty, Q1} is returned.

Example:

1> Queue = queue:from_list([1,2,3,4,5]).
{[5,4,3],[1,2]}
2> {{value, 1=Item}, Queue1} = queue:out(Queue).
{{value,1},{[5,4,3],[2]}}
3> queue:to_list(Queue1).
[2,3,4,5]

out_r(Q1)

-spec out_r(Q1 :: queue(Item)) -> {{value, Item}, Q2 :: queue(Item)} | {empty, Q1 :: queue(Item)}.

Removes the item at the rear of queue Q1. Returns tuple {{value, Item}, Q2}, where Item is the item removed and Q2 is the new queue. If Q1 is empty, tuple {empty, Q1} is returned.

Example:

1> Queue = queue:from_list([1,2,3,4,5]).
{[5,4,3],[1,2]}
2> {{value, 5=Item}, Queue1} = queue:out_r(Queue).
{{value,5},{[4,3],[1,2]}}
3> queue:to_list(Queue1).
[1,2,3,4]

reverse(Q1)

-spec reverse(Q1 :: queue(Item)) -> Q2 :: queue(Item).

Returns a queue Q2 containing the items of Q1 in the reverse order.

split(N, Q1)

-spec split(N :: non_neg_integer(), Q1 :: queue(Item)) -> {Q2 :: queue(Item), Q3 :: queue(Item)}.

Splits Q1 in two. The N front items are put in Q2 and the rest in Q3.

to_list(Q)

-spec to_list(Q :: queue(Item)) -> [Item].

Returns a list of the items in the queue in the same order; the front item of the queue becomes the head of the list.

Example:

1> Queue = queue:from_list([1,2,3,4,5]).
{[5,4,3],[1,2]}
2> List == queue:to_list(Queue).
true

sets

Sets are collections of elements with no duplicate elements.

The data representing a set as used by this module is to be regarded as opaque by other modules. In abstract terms, the representation is a composite type of existing Erlang terms. See note on data types. Any code assuming knowledge of the format is running on thin ice.

This module provides the same interface as the ordsets module but with an undefined representation. One key difference is that this module considers two elements as different if they do not match (=:=), whereas ordsets considers them different if and only if they do not compare equal (==).

Erlang/OTP 24.0 introduced a new more performant representation for sets which has become the default in Erlang/OTP 28. Developers can use the old representation by passing the {version, 1} flag to new/1 and from_list/2. Functions that work on two sets, such as union/2, will work with sets of different versions. In such cases, there is no guarantee about the version of the returned set. Explicit conversion from the old version to the new one can be done with sets:from_list(sets:to_list(Old), [{version,2}]).

Compatibility

The following functions in this module also exist and provide the same functionality in the gb_sets and ordsets modules. That is, by only changing the module name for each call, you can try out different set representations.

add_element/2

del_element/2

filter/2

filtermap/2

fold/3

from_list/1

intersection/1

intersection/2

is_element/2

is_empty/1

is_equal/2

is_set/1

is_subset/2

map/2

new/0

size/1

subtract/2

to_list/1

union/1

union/2

Note

While the three set implementations offer the same functionality with respect to the aforementioned functions, their overall behavior may differ. As mentioned, this module considers elements as different if and only if they do not match (=:=), while both ordsets and gb_sets consider elements as different if and only if they do not compare equal (==).

Examples

1> sets:is_element(1.0, sets:from_list([1])).
false
2> ordsets:is_element(1.0, ordsets:from_list([1])).
true
3> gb_sets:is_element(1.0, gb_sets:from_list([1])).
true

See Also

gb_sets, ordsets

Summary

Types

set()

set(Element)

As returned by new/0.

Functions

add_element(Element, Set1)

Returns a new set formed from Set1 with Element inserted.

del_element(Element, Set1)

Returns a copy of Set1 with Element removed.

filter(Pred, Set1)

Filters elements in Set1 using predicate function Pred.

filtermap(Fun, Set1)

Calls Fun(Elem) for each Elem of Set1 to update or remove elements from Set1.

fold(Function, Acc0, Set)

Folds Function over every element in Set and returns the final value of the accumulator.

from_list(List)

Returns a set of the elements in List.

from_list/2

Returns a set of the elements in List of the given version.

intersection(SetList)

Returns the intersection of the non-empty list of sets.

intersection(Set1, Set2)

Returns the intersection of Set1 and Set2.

is_disjoint(Set1, Set2)

Returns true if Set1 and Set2 are disjoint; otherwise, returns false.

is_element(Element, Set)

Returns true if Element is an element of Set; otherwise, returns false.

is_empty(Set)

Returns true if Set is an empty set; otherwise, returns false.

is_equal(Set1, Set2)

Returns true if Set1 and Set2 are equal, that is, if every element of one set is also a member of the other set; otherwise, returns false.

is_set(Set)

Returns true if Set appears to be a set of elements; otherwise, returns false.

is_subset(Set1, Set2)

Returns true when every element of Set1 is also a member of Set2; otherwise, returns false.

map(Fun, Set1)

Maps elements in Set1 with mapping function Fun.

new()

Returns a new empty set.

new/1

Returns a new empty set of the given version.

size(Set)

Returns the number of elements in Set.

subtract(Set1, Set2)

Returns a new set containing the elements of Set1 that are not elements in Set2.

to_list(Set)

Returns the elements of Set as a list.

union(SetList)

Returns the union of a list of sets.

union(Set1, Set2)

Returns the union of Set1 and Set2.

Types

set()

-type set() :: set(_).

set(Element)

-opaque set(Element)

As returned by new/0.

Functions

add_element(Element, Set1)

-spec add_element(Element, Set1) -> Set2 when Set1 :: set(Element), Set2 :: set(Element).

Returns a new set formed from Set1 with Element inserted.

Examples

1> S0 = sets:new().
2> S1 = sets:add_element(7, S0).
3> sets:to_list(S1).
[7]
4> S2 = sets:add_element(42, S1).
5> lists:sort(sets:to_list(S2)).
[7,42]
6> S2 = sets:add_element(42, S1).
7> lists:sort(sets:to_list(S2)).
[7,42]

del_element(Element, Set1)

-spec del_element(Element, Set1) -> Set2 when Set1 :: set(Element), Set2 :: set(Element).

Returns a copy of Set1 with Element removed.

Examples

1> S = sets:from_list([a,b]).
2> sets:to_list(sets:del_element(b, S)).
[a]
3> S = sets:del_element(x, S).
4> lists:sort(sets:to_list(S)).
[a,b]

filter(Pred, Set1)

-spec filter(Pred, Set1) -> Set2
                when Pred :: fun((Element) -> boolean()), Set1 :: set(Element), Set2 :: set(Element).

Filters elements in Set1 using predicate function Pred.

Examples

1> S = sets:from_list([1,2,3,4,5,6,7]).
2> IsEven = fun(N) -> N rem 2 =:= 0 end.
3> Filtered = sets:filter(IsEven, S).
4> lists:sort(sets:to_list(Filtered)).
[2,4,6]

filtermap(Fun, Set1)

(since OTP 27.0)

-spec filtermap(Fun, Set1) -> Set2
                   when
                       Fun :: fun((Element1) -> boolean() | {true, Element2}),
                       Set1 :: set(Element1),
                       Set2 :: set(Element1 | Element2).

Calls Fun(Elem) for each Elem of Set1 to update or remove elements from Set1.

Fun/1 must return either a Boolean or a tuple {true, Value}. The function returns the set of elements for which Fun returns a new value, with true being equivalent to {true, Elem}.

sets:filtermap/2 behaves as if it were defined as follows:

filtermap(Fun, Set1) ->
    sets:from_list(lists:filtermap(Fun, Set1)).

Examples

1> S = sets:from_list([2,4,5,6,8,9])
2> F = fun(X) ->
           case X rem 2 of
               0 -> {true, X div 2};
               1 -> false
           end
        end.
3> Set = sets:filtermap(F, S).
4> lists:sort(sets:to_list(Set)).
[1,2,3,4]

fold(Function, Acc0, Set)

-spec fold(Function, Acc0, Set) -> Acc1
              when
                  Function :: fun((Element, AccIn) -> AccOut),
                  Set :: set(Element),
                  Acc0 :: Acc,
                  Acc1 :: Acc,
                  AccIn :: Acc,
                  AccOut :: Acc.

Folds Function over every element in Set and returns the final value of the accumulator.

The evaluation order is undefined.

Examples

1> S = sets:from_list([1,2,3,4]).
2> Plus = fun erlang:'+'/2.
3> sets:fold(Plus, 0, S).
10

from_list(List)

-spec from_list(List) -> Set when List :: [Element], Set :: set(Element).

Returns a set of the elements in List.

Examples

1> S = sets:from_list([a,b,c]).
2> lists:sort(sets:to_list(S)).
[a,b,c]

from_list/2

(since OTP 24.0)

-spec from_list(List, [{version, 1..2}]) -> Set when List :: [Element], Set :: set(Element).

Returns a set of the elements in List of the given version.

Examples

1> S = sets:from_list([a,b,c], [{version, 1}]).
2> lists:sort(sets:to_list(S)).
[a,b,c]

intersection(SetList)

-spec intersection(SetList) -> Set when SetList :: [set(Element), ...], Set :: set(Element).

Returns the intersection of the non-empty list of sets.

The intersection of multiple sets is a new set that contains only the elements that are present in all sets.

Examples

1> S0 = sets:from_list([a,b,c,d]).
2> S1 = sets:from_list([d,e,f]).
3> S2 = sets:from_list([q,r])
4> Sets = [S0, S1, S2].
5> sets:to_list(sets:intersection([S0, S1, S2])).
[]
6> sets:to_list(sets:intersection([S0, S1])).
[d]
7> sets:intersection([]).
** exception error: no function clause matching sets:intersection([])

intersection(Set1, Set2)

-spec intersection(Set1, Set2) -> Set3
                      when Set1 :: set(Element), Set2 :: set(Element), Set3 :: set(Element).

Returns the intersection of Set1 and Set2.

The intersection of two sets is a new set that contains only the elements that are present in both sets.

Examples

1> S0 = sets:from_list([a,b,c,d]).
2> S1 = sets:from_list([c,d,e,f]).
3> S2 = sets:from_list([q,r]).
4> Intersection = sets:intersection(S0, S1).
5> lists:sort(sets:to_list(Intersection)).
[c,d]
6> sets:to_list(sets:intersection(S1, S2)).
[]

is_disjoint(Set1, Set2)

-spec is_disjoint(Set1, Set2) -> boolean() when Set1 :: set(Element), Set2 :: set(Element).

Returns true if Set1 and Set2 are disjoint; otherwise, returns false.

Two sets are disjoint if they have no elements in common.

This function is equivalent to sets:intersection(Set1, Set2) =:= [], but faster.

Examples

1> S0 = sets:from_list([a,b,c,d]).
2> S1 = sets:from_list([d,e,f]).
3> S2 = sets:from_list([q,r])
4> sets:is_disjoint(S0, S1).
false
5> sets:is_disjoint(S1, S2).
true

is_element(Element, Set)

-spec is_element(Element, Set) -> boolean() when Set :: set(Element).

Returns true if Element is an element of Set; otherwise, returns false.

Examples

1> S = sets:from_list([a,b,c]).
2> sets:is_element(42, S).
false
3> sets:is_element(b, S).
true

is_empty(Set)

(since OTP 21.0)

-spec is_empty(Set) -> boolean() when Set :: set().

Returns true if Set is an empty set; otherwise, returns false.

Examples

1> sets:is_empty(sets:new()).
true
2> sets:is_empty(sets:from_list([1])).
false

is_equal(Set1, Set2)

(since OTP 27.0)

-spec is_equal(Set1, Set2) -> boolean() when Set1 :: set(), Set2 :: set().

Returns true if Set1 and Set2 are equal, that is, if every element of one set is also a member of the other set; otherwise, returns false.

Examples

1> Empty = sets:new().
2> S = sets:from_list([a,b]).
3> sets:is_equal(S, S)
true
4> sets:is_equal(S, Empty)
false
5> OldSet = sets:from_list([a,b], [{version,1}]).
6> sets:is_equal(S, OldSet).
true
7> S =:= OldSet.
false

is_set(Set)

-spec is_set(Set) -> boolean() when Set :: term().

Returns true if Set appears to be a set of elements; otherwise, returns false.

Note

Note that the test is shallow and will return true for any term that coincides with the possible representations of a set. See also note on data types.

Furthermore, since sets are opaque, calling this function on terms that are not sets could result in dialyzer warnings.

Examples

1> sets:is_set(sets:new()).
true
2> sets:is_set(sets:new([{version,1}])).
true
3> sets:is_set(0).
false

is_subset(Set1, Set2)

-spec is_subset(Set1, Set2) -> boolean() when Set1 :: set(Element), Set2 :: set(Element).

Returns true when every element of Set1 is also a member of Set2; otherwise, returns false.

Examples

1> S0 = sets:from_list([a,b,c,d]).
2> S1 = sets:from_list([c,d]).
3> sets:is_subset(S1, S0).
true
4> sets:is_subset(S0, S1).
false
5> sets:is_subset(S0, S0).
true

map(Fun, Set1)

(since OTP 27.0)

-spec map(Fun, Set1) -> Set2
             when Fun :: fun((Element1) -> Element2), Set1 :: set(Element1), Set2 :: set(Element2).

Maps elements in Set1 with mapping function Fun.

Examples

1> S = sets:from_list([1,2,3,4,5,6,7]).
2> F = fun(N) -> N div 2 end.
3> Mapped = sets:map(F, S).
4> lists:sort(sets:to_list(Mapped)).
[0,1,2,3]

new()

-spec new() -> set(none()).

Returns a new empty set.

Examples

1> sets:to_list(sets:new()).
[]

new/1

(since OTP 24.0)

-spec new([{version, 1..2}]) -> set(none()).

Returns a new empty set of the given version.

Examples

1> sets:to_list(sets:new([{version, 1}])).
[]
2> sets:new() =:= sets:new([{version, 2}]).
true

size(Set)

-spec size(Set) -> non_neg_integer() when Set :: set().

Returns the number of elements in Set.

Examples

1> sets:size(sets:new()).
0
2> sets:size(sets:from_list([4,5,6])).
3

subtract(Set1, Set2)

-spec subtract(Set1, Set2) -> Set3 when Set1 :: set(Element), Set2 :: set(Element), Set3 :: set(Element).

Returns a new set containing the elements of Set1 that are not elements in Set2.

Examples

1> S0 = sets:from_list([a,b,c,d]).
2> S1 = sets:from_list([c,d,e,f]).
3> lists:sort(sets:to_list(sets:subtract(S0, S1))).
[a,b]
4> lists:sort(sets:to_list(sets:subtract(S1, S0))).
[e,f]

to_list(Set)

-spec to_list(Set) -> List when Set :: set(Element), List :: [Element].

Returns the elements of Set as a list.

The order of the returned elements is undefined.

Examples

1> S = sets:from_list([1,2,3]).
2> lists:sort(sets:to_list(S)).
[1,2,3]

union(SetList)

-spec union(SetList) -> Set when SetList :: [set(Element)], Set :: set(Element).

Returns the union of a list of sets.

The union of multiple sets is a new set that contains all the elements from all sets, without duplicates.

Examples

1> S0 = sets:from_list([a,b,c,d]).
2> S1 = sets:from_list([d,e,f]).
3> S2 = sets:from_list([q,r])
4> Sets = [S0, S1, S2].
5> Union = sets:union(Sets).
6> lists:sort(sets:to_list(Union)).
[a,b,c,d,e,f,q,r]

union(Set1, Set2)

-spec union(Set1, Set2) -> Set3 when Set1 :: set(Element), Set2 :: set(Element), Set3 :: set(Element).

Returns the union of Set1 and Set2.

The union of two sets is a new set that contains all the elements from both sets, without duplicates.

Examples

1> S0 = sets:from_list([a,b,c,d]).
2> S1 = sets:from_list([c,d,e,f]).
3> Union = sets:union(S0, S1).
4> lists:sort(sets:to_list(Union)).
[a,b,c,d,e,f]

sofs

Functions for manipulating sets of sets.

This module provides operations on finite sets and relations represented as sets. Intuitively, a set is a collection of elements; every element belongs to the set, and the set contains every element.

The data representing sofs as used by this module is to be regarded as opaque by other modules. In abstract terms, the representation is a composite type of existing Erlang terms. See note on data types. Any code assuming knowledge of the format is running on thin ice.

Getting started

A recommended starting point for the first-time user is the examples for the following functions:

relation_to_family/1

restriction/2 and drestriction/2

image/2 and inverse_image/2

converse/1

Set theory

Given a set A and a sentence S(x), where x is a free variable, a new set B whose elements are exactly those elements of A for which S(x) holds can be formed, this is denoted B = {x in A : S(x)}. Sentences are expressed using the logical operators "for some" (or "there exists"), "for all", "and", "or", "not". If the existence of a set containing all the specified elements is known (as is always the case in this module), this is denoted B = {x : S(x)}.

The unordered set containing the elements a, b, and c is denoted {a, b, c}. This notation is not to be confused with tuples.

The ordered pair of a and b, with first coordinate a and second coordinate b, is denoted (a, b). An ordered pair is an ordered set of two elements. In this module, ordered sets can contain one, two, or more elements, and parentheses are used to enclose the elements.

Unordered sets and ordered sets are orthogonal, again in this module; there is no unordered set equal to any ordered set.

The empty set contains no elements.

Set A is equal to set B if they contain the same elements, which is denoted A = B. Two ordered sets are equal if they contain the same number of elements and have equal elements at each coordinate.

Set B is a subset of set A if A contains all elements that B contains.

The union of two sets A and B is the smallest set that contains all elements of A and all elements of B.

The intersection of two sets A and B is the set that contains all elements of A that belong to B.

Two sets are disjoint if their intersection is the empty set.

The difference of two sets A and B is the set that contains all elements of A that do not belong to B.

The symmetric difference of two sets is the set that contains those element that belong to either of the two sets, but not both.

The union of a collection of sets is the smallest set that contains all the elements that belong to at least one set of the collection.

The intersection of a non-empty collection of sets is the set that contains all elements that belong to every set of the collection.

The Cartesian product of two sets X and Y, denoted X × Y, is the set {a : a = (x, y) for some x in X and for some y in Y}.

A relation is a subset of X × Y. Let R be a relation. The fact that (x, y) belongs to R is written as x R y. As relations are sets, the definitions of the last item (subset, union, and so on) apply to relations as well.

The domain of R is the set {x : x R y for some y in Y}.

The range of R is the set {y : x R y for some x in X}.

The converse of R is the set {a : a = (y, x) for some (x, y) in R}.

If A is a subset of X, the image of A under R is the set {y : x R y for some x in A}. If B is a subset of Y, the inverse image of B is the set {x : x R y for some y in B}.

If R is a relation from X to Y, and S is a relation from Y to Z, the relative product of R and S is the relation T from X to Z defined so that x T z if and only if there exists an element y in Y such that x R y and y S z.

The restriction of R to A is the set S defined so that x S y if and only if there exists an element x in A such that x R y.

If S is a restriction of R to A, then R is an extension of S to X.

If X = Y, then R is called a relation in X.

The field of a relation R in X is the union of the domain of R and the range of R.

If R is a relation in X, and if S is defined so that x S y if x R y and not x = y, then S is the strict relation corresponding to R. Conversely, if S is a relation in X, and if R is defined so that x R y if x S y or x = y, then R is the weak relation corresponding to S.

A relation R in X is reflexive if x R x for every element x of X, it is symmetric if x R y implies that y R x, and it is transitive if x R y and y R z imply that x R z.

A function F is a relation, a subset of X × Y, such that the domain of F is equal to X and such that for every x in X there is a unique element y in Y with (x, y) in F. The latter condition can be formulated as follows: if x F y and x F z, then y = z. In this module, it is not required that the domain of F is equal to X for a relation to be considered a function.

Instead of writing (x, y) in F or x F y, we write F(x) = y when F is a function, and say that F maps x onto y, or that the value of F at x is y.

As functions are relations, the definitions of the last item (domain, range, and so on) apply to functions as well.

If the converse of a function F is a function F', then F' is called the inverse of F.

The relative product of two functions F1 and F2 is called the composite of F1 and F2 if the range of F1 is a subset of the domain of F2.

Sometimes, when the range of a function is more important than the function itself, the function is called a family.

The domain of a family is called the index set, and the range is called the indexed set.

If x is a family from I to X, then x[i] denotes the value of the function at index i. The notation "a family in X" is used for such a family.

When the indexed set is a set of subsets of a set X, we call x a family of subsets of X.

If x is a family of subsets of X, the union of the range of x is called the union of the family x.

If x is non-empty (the index set is non-empty), the intersection of the family x is the intersection of the range of x.

In this module, the only families that are considered are families of subsets of some set X; in the following, the word "family" is used for such families of subsets.

A partition of a set X is a collection S of non-empty subsets of X whose union is X and whose elements are pairwise disjoint.

A relation in a set is an equivalence relation if it is reflexive, symmetric, and transitive.

If R is an equivalence relation in X, and x is an element of X, the equivalence class of x with respect to R is the set of all those elements y of X for which x R y holds. The equivalence classes constitute a partitioning of X. Conversely, if C is a partition of X, the relation that holds for any two elements of X if they belong to the same equivalence class, is an equivalence relation induced by the partition C.

If R is an equivalence relation in X, the canonical map is the function that maps every element of X onto its equivalence class.

Relations as defined above (as sets of ordered pairs) are from now on referred to as binary relations.

We call a set of ordered sets (x[1], ..., x[n]) an (n-ary) relation, and say that the relation is a subset of the Cartesian product X[1] × ... × X[n], where x[i] is an element of X[i], 1 <= i <= n.

The projection of an n-ary relation R onto coordinate i is the set {x[i] : (x[1], ..., x[i], ..., x[n]) in R for some x[j] in X[j], 1 <= j <= n and not i = j}. The projections of a binary relation R onto the first and second coordinates are the domain and the range of R, respectively.

The relative product of binary relations can be generalized to n-ary relations as follows. Let TR be an ordered set (R[1], ..., R[n]) of binary relations from X to Y[i] and S a binary relation from (Y[1] × ... × Y[n]) to Z. The relative product of TR and S is the binary relation T from X to Z defined so that x T z if and only if there exists an element y[i] in Y[i] for each 1 <= i <= n such that x R[i] y[i] and (y[1], ..., y[n]) S z. Now let TR be a an ordered set (R[1], ..., R[n]) of binary relations from X[i] to Y[i] and S a subset of X[1] × ... × X[n].

The multiple relative product of TR and S is defined to be the set {z : z = ((x[1], ..., x[n]), (y[1],...,y[n])) for some (x[1], ..., x[n]) in S and for some (x[i], y[i]) in R[i], 1 <= i <= n}.

The natural join of an n-ary relation R and an m-ary relation S on coordinate i and j is defined to be the set {z : z = (x[1], ..., x[n],  y[1], ..., y[j-1], y[j+1], ..., y[m]) for some (x[1], ..., x[n]) in R and for some (y[1], ..., y[m]) in S such that x[i] = y[j]}.

Sets handled by this module

The sets recognized by this module are represented by elements of the relation Sets, which is defined as the smallest set such that:

For every atom T, except '_', and for every term X, (T, X) belongs to Sets (atomic sets).

(['_'], []) belongs to Sets (the untyped empty set).

For every tuple T = {T[1], ..., T[n]} and for every tuple X = {X[1], ..., X[n]}, if (T[i], X[i]) belongs to Sets for every 1 <= i <= n, then (T, X) belongs to Sets (ordered sets).

For every term T, if X is the empty list or a non-empty sorted list [X[1], ..., X[n]] without duplicates such that (T, X[i]) belongs to Sets for every 1 <= i <= n, then ([T], X) belongs to Sets (typed unordered sets).

An external set is an element of the range of Sets.

A type is an element of the domain of Sets.

If S is an element (T, X) of Sets, then T is a valid type of X, T is the type of S, and X is the external set of S. from_term/2 creates a set from a type and an Erlang term turned into an external set.

The sets represented by Sets are the elements of the range of function Set from Sets to Erlang terms and sets of Erlang terms:

Set(T,Term) = Term, where T is an atom

Set({T[1], ..., T[n]}, {X[1], ...,  X[n]}) = (Set(T[1], X[1]), ...,  Set(T[n], X[n]))

Set([T], [X[1], ..., X[n]]) = {Set(T, X[1]), ..., Set(T, X[n])}

Set([T], []) = {}

When there is no risk of confusion, elements of Sets are identified with the sets they represent. For example, if U is the result of calling union/2 with S1 and S2 as arguments, then U is said to be the union of S1 and S2. A more precise formulation is that Set(U) is the union of Set(S1) and Set(S2).

The types are used to implement the various conditions that sets must fulfill. As an example, consider the relative product of two sets R and S, and recall that the relative product of R and S is defined if R is a binary relation to Y and S is a binary relation from Y. The function that implements the relative product, relative_product/2, checks that the arguments represent binary relations by matching [{A,B}] against the type of the first argument (Arg1 say), and [{C,D}] against the type of the second argument (Arg2 say). The fact that [{A,B}] matches the type of Arg1 is to be interpreted as Arg1 representing a binary relation from X to Y, where X is defined as all sets Set(x) for some element x in Sets the type of which is A, and similarly for Y. In the same way Arg2 is interpreted as representing a binary relation from W to Z. Finally it is checked that B matches C, which is sufficient to ensure that W is equal to Y. The untyped empty set is handled separately: its type, ['_'], matches the type of any unordered set.

A few functions of this module (drestriction/3, family_projection/2, partition/2, partition_family/2, projection/2, restriction/3, substitution/2) accept an Erlang function as a means to modify each element of a given unordered set. Such a function, called SetFun in the following, can be specified as a functional object (fun), a tuple {external, Fun}, or an integer:

If SetFun is specified as a fun, the fun is applied to each element of the given set and the return value is assumed to be a set.

If SetFun is specified as a tuple {external, Fun}, Fun is applied to the external set of each element of the given set and the return value is assumed to be an external set. Selecting the elements of an unordered set as external sets and assembling a new unordered set from a list of external sets is in the present implementation more efficient than modifying each element as a set. However, this optimization can only be used when the elements of the unordered set are atomic or ordered sets. It must also be the case that the type of the elements matches some clause of Fun (the type of the created set is the result of applying Fun to the type of the given set), and that Fun does nothing but selecting, duplicating, or rearranging parts of the elements.

Specifying a SetFun as an integer I is equivalent to specifying {external, fun(X) -> element(I, X) end}, but is to be preferred, as it makes it possible to handle this case even more efficiently.

Examples of valid SetFuns:

fun sofs:union/1
fun(S) -> sofs:partition(1, S) end
fun(S) -> sofs:from_term(sofs:no_elements(S)) end
{external, fun(A) -> A end}
{external, fun({A,_,C}) -> {C,A} end}
{external, fun({_,{_,C}}) -> C end}
{external, fun({_,{_,{_,E}=C}}) -> {E,{E,C}} end}
2

Examples of invalid SetFuns:

fun sofs:no_elements/1
{external, fun(A) -> 2 * A end}
{external, fun({A,B,C}) -> A + B + C end}
{external, fun lists:sum/1}

The order in which a SetFun is applied to the elements of an unordered set is not specified, and can change in future versions of this module.

The execution time of the functions of this module is dominated by the time it takes to sort lists. When no sorting is needed, the execution time is in the worst case proportional to the sum of the sizes of the input arguments and the returned value. A few functions execute in constant time: from_external/2, is_empty_set/1, is_set/1, is_sofs_set/1, to_external/1 type/1.

The functions of this module exit the process with a badarg, bad_function, or type_mismatch message when given badly formed arguments or sets the types of which are not compatible.

When comparing external sets, operator ==/2 is used.

See Also

digraph, gb_sets, gb_trees, maps, orddict, ordsets, sets

Summary

Types

a_function()

A function.

a_set()

An unordered set.

anyset()

Any kind of set (also included are the atomic sets).

binary_relation()

A binary relation.

external_set()

An external set.

family()

A family (of subsets).

ordset()

An ordered set.

relation()

An n-ary relation.

set_fun()

A SetFun.

set_of_sets()

An unordered set of unordered sets.

spec_fun()

tuple_of(T)

A tuple where the elements are of type T.

type()

A type.

Functions

a_function(Tuples)

Equivalent to a_function(Tuples, [{atom, atom}]).

a_function(Tuples, Type)

Creates a function.

canonical_relation(SetOfSets)

Returns the binary relation containing the elements (E, Set) such that Set belongs to SetOfSets and E belongs to Set.

composite(Function1, Function2)

Returns the composite of the functions Function1 and Function2.

constant_function(Set, AnySet)

Creates the function that maps each element of set Set onto AnySet.

converse(BinRel1)

Returns the converse of the binary relation BinRel1.

difference(Set1, Set2)

Returns the difference of the sets Set1 and Set2.

digraph_to_family(Graph)

Equivalent to digraph_to_family(Graph, [{atom, [atom]}]).

digraph_to_family(Graph, Type)

Creates a family from the directed graph Graph.

domain(BinRel)

Returns the domain of the binary relation BinRel.

drestriction(BinRel1, Set)

Returns the difference between the binary relation BinRel1 and the restriction of BinRel1 to Set.

drestriction(SetFun, Set1, Set2)

Returns a subset of Set1 containing those elements that do not give an element in Set2 as the result of applying SetFun.

empty_set()

Returns the untyped empty set.

extension(BinRel1, Set, AnySet)

Returns the extension of BinRel1 such that for each element E in Set that does not belong to the domain of BinRel1, BinRel2 contains the pair (E, AnySet).

family(Tuples)

Equivalent to family(Tuples, [{atom, [atom]}]).

family(Tuples, Type)

Creates a family of subsets.

family_difference(Family1, Family2)

If Family1 and Family2 are families, then Family3 is the family such that the index set is equal to the index set of Family1, and Family3[i] is the difference between Family1[i] and Family2[i] if Family2 maps i, otherwise Family1[i].

family_domain(Family1)

If Family1 is a family and Family1[i] is a binary relation for every i in the index set of Family1, then Family2 is the family with the same index set as Family1 such that Family2[i] is the domain of Family1[i].

family_field(Family1)

If Family1 is a family and Family1[i] is a binary relation for every i in the index set of Family1, then Family2 is the family with the same index set as Family1 such that Family2[i] is the field of Family1[i].

family_intersection(Family1)

If Family1 is a family and Family1[i] is a set of sets for every i in the index set of Family1, then Family2 is the family with the same index set as Family1 such that Family2[i] is the intersection of Family1[i].

family_intersection(Family1, Family2)

If Family1 and Family2 are families, then Family3 is the family such that the index set is the intersection of Family1:s and Family2:s index sets, and Family3[i] is the intersection of Family1[i] and Family2[i].

family_projection(SetFun, Family1)

If Family1 is a family, then Family2 is the family with the same index set as Family1 such that Family2[i] is the result of calling SetFun with Family1[i] as argument.

family_range(Family1)

If Family1 is a family and Family1[i] is a binary relation for every i in the index set of Family1, then Family2 is the family with the same index set as Family1 such that Family2[i] is the range of Family1[i].

family_specification(Fun, Family1)

If Family1 is a family, then Family2 is the restriction of Family1 to those elements i of the index set for which Fun applied to Family1[i] returns true.

family_to_digraph(Family)

Equivalent to family_to_digraph(Family, []).

family_to_digraph(Family, GraphType)

Creates a directed graph from family Family.

family_to_relation(Family)

If Family is a family, then BinRel is the binary relation containing all pairs (i, x) such that i belongs to the index set of Family and x belongs to Family[i].

family_union(Family1)

If Family1 is a family and Family1[i] is a set of sets for each i in the index set of Family1, then Family2 is the family with the same index set as Family1 such that Family2[i] is the union of Family1[i].

family_union(Family1, Family2)

If Family1 and Family2 are families, then Family3 is the family such that the index set is the union of Family1:s and Family2:s index sets, and Family3[i] is the union of Family1[i] and Family2[i] if both map i, otherwise Family1[i] or Family2[i].

field(BinRel)

Returns the field of the binary relation BinRel.

from_external(ExternalSet, Type)

Creates a set from the external set ExternalSet and the type Type.

from_sets(AnySet)

Returns the unordered set containing the sets of list ListOfSets, or returns the ordered set containing the sets of the non-empty tuple TupleOfSets.

from_term(Term)

Equivalent to from_term(Term, '_').

from_term(Term, Type)

Creates an element of Sets by traversing term Term, sorting lists, removing duplicates, and deriving or verifying a valid type for the so obtained external set.

image(BinRel, Set1)

Returns the image of set Set1 under the binary relation BinRel.

intersection(SetOfSets)

Returns the intersection of the set of sets SetOfSets.

intersection(Set1, Set2)

Returns the intersection of Set1 and Set2.

intersection_of_family(Family)

Returns the intersection of family Family.

inverse(Function1)

Returns the inverse of function Function1.

inverse_image(BinRel, Set1)

Returns the inverse image of Set1 under the binary relation BinRel.

is_a_function(BinRel)

Returns true if the binary relation BinRel is a function or the untyped empty set; otherwise, returns false.

is_disjoint(Set1, Set2)

Returns true if Set1 and Set2 are disjoint; otherwise, returns false.

is_empty_set(AnySet)

Returns true if AnySet is an empty unordered set; otherwise, returns false.

is_equal(AnySet1, AnySet2)

Returns true if AnySet1 and AnySet2 are equal, otherwise false.

is_set(AnySet)

Returns true if AnySet appears to be an unordered set, and false if AnySet is an ordered set or an atomic set or any other term.

is_sofs_set(Term)

Returns true if Term appears to be an unordered set, an ordered set, or an atomic set; otherwise, returns false.

is_subset(Set1, Set2)

Returns true if Set1 is a subset of Set2; otherwise, returns false.

is_type(Term)

Returns true if term Term is a type.

join(Relation1, I, Relation2, J)

Returns the natural join of the relations Relation1 and Relation2 on coordinates I and J.

multiple_relative_product(TupleOfBinRels, BinRel1)

If TupleOfBinRels is a non-empty tuple {R[1], ..., R[n]} of binary relations and BinRel1 is a binary relation, then BinRel2 is the multiple relative product of the ordered set (R[i], ..., R[n]) and BinRel1.

no_elements(ASet)

Returns the number of elements of the ordered or unordered set ASet.

partition(SetOfSets)

Returns the partition of the union of the set of sets SetOfSets such that two elements are considered equal if they belong to the same elements of SetOfSets.

partition(SetFun, Set)

Returns the partition of Set such that two elements are considered equal if the results of applying SetFun are equal.

partition(SetFun, Set1, Set2)

Returns a pair of sets that, regarded as constituting a set, forms a partition of Set1.

partition_family(SetFun, Set)

Returns family Family where the indexed set is a partition of Set such that two elements are considered equal if the results of applying SetFun are the same value i.

product(TupleOfSets)

Returns the Cartesian product of the non-empty tuple of sets TupleOfSets.

product(Set1, Set2)

Returns the Cartesian product of Set1 and Set2.

projection(SetFun, Set1)

Returns the set created by substituting each element of Set1 by the result of applying SetFun to the element.

range(BinRel)

Returns the range of the binary relation BinRel.

relation(Tuples)

Equivalent to relation(Tuples, Type), where Type is the size of the first tuple of Tuples, if such a tuple exists.

relation(Tuples, Type)

Creates a relation.

relation_to_family(BinRel)

Returns family Family such that the index set is equal to the domain of the binary relation BinRel, and Family[i] is the image of the set of i under BinRel.

relative_product1(BinRel1, BinRel2)

Returns the relative product of the converse of the binary relation BinRel1 and the binary relation BinRel2.

relative_product(ListOfBinRels)

Returns relative product of the ordered set (R[i], ..., R[n]) and the relation of equality between the elements of the Cartesian product of the ranges of R[i], range R[1] × ... × range R[n].

relative_product(ListOrRel, BinRel1)

Returns the relative product.

restriction(BinRel1, Set)

Returns the restriction of the binary relation BinRel1 to Set.

restriction(SetFun, Set1, Set2)

Returns a subset of Set1 containing those elements that gives an element in Set2 as the result of applying SetFun.

set(Terms)

Equivalent to set(Terms, [atom]).

set(Terms, Type)

Creates an unordered set.

specification(Fun, Set1)

Returns the set containing every element of Set1 for which Fun returns true.

strict_relation(BinRel1)

Returns the strict relation corresponding to the binary relation BinRel1.

substitution(SetFun, Set1)

Returns a function with the domain Set1, where each element maps to the result of applying SetFun to it.

symdiff(Set1, Set2)

Returns the symmetric difference (or the Boolean sum) of Set1 and Set2.

symmetric_partition(Set1, Set2)

Returns the symmetric partition of Set1 and Set2.

to_external(AnySet)

Returns the external set of an atomic, ordered, or unordered set.

to_sets(ASet)

Returns the elements of the ordered set ASet as a tuple of sets, and the elements of the unordered set ASet as a sorted list of sets without duplicates.

type(AnySet)

Returns the type of an atomic, ordered, or unordered set.

union(SetOfSets)

Returns the union of the set of sets SetOfSets.

union(Set1, Set2)

Returns the union of Set1 and Set2.

union_of_family(Family)

Returns the union of family Family.

weak_relation(BinRel1)

Returns a subset S of the weak relation W corresponding to the binary relation BinRel1.

Types

a_function()

-type a_function() :: relation().

A function.

a_set()

-opaque a_set()

An unordered set.

anyset()

-type anyset() :: ordset() | a_set().

Any kind of set (also included are the atomic sets).

binary_relation()

-type binary_relation() :: relation().

A binary relation.

external_set()

-type external_set() :: term().

An external set.

family()

-type family() :: a_function().

A family (of subsets).

ordset()

-opaque ordset()

An ordered set.

relation()

-type relation() :: a_set().

An n-ary relation.

set_fun()

-type set_fun() ::
          pos_integer() |
          {external, fun((external_set()) -> external_set())} |
          fun((anyset()) -> anyset()).

A SetFun.

set_of_sets()

-type set_of_sets() :: a_set().

An unordered set of unordered sets.

spec_fun()

-type spec_fun() :: {external, fun((external_set()) -> boolean())} | fun((anyset()) -> boolean()).

tuple_of(T)

(not exported)

-type tuple_of(_T) :: tuple().

A tuple where the elements are of type T.

type()

-type type() :: term().

A type.

Functions

a_function(Tuples)

-spec a_function(Tuples) -> Function when Function :: a_function(), Tuples :: [tuple()].

Equivalent to a_function(Tuples, [{atom, atom}]).

a_function(Tuples, Type)

-spec a_function(Tuples, Type) -> Function
                    when Function :: a_function(), Tuples :: [tuple()], Type :: type().

Creates a function.

a_function(F, T) is equivalent to from_term(F, T) if the result is a function.

Examples

1> sofs:is_a_function(sofs:a_function([{1,a},{2,b},{3,c}])).
true
2> sofs:a_function([{1,a},{1,b}]).
** exception error: bad_function
     in function  sofs:a_function/1

canonical_relation(SetOfSets)

-spec canonical_relation(SetOfSets) -> BinRel
                            when BinRel :: binary_relation(), SetOfSets :: set_of_sets().

Returns the binary relation containing the elements (E, Set) such that Set belongs to SetOfSets and E belongs to Set.

If SetOfSets is a partition of a set X and R is the equivalence relation in X induced by SetOfSets, then the returned relation is the canonical map from X onto the equivalence classes with respect to R.

Examples

1> Ss = sofs:from_term([[a,b],[b,c]]).
2> CR = sofs:canonical_relation(Ss).
3> sofs:to_external(CR).
[{a,[a,b]},{b,[a,b]},{b,[b,c]},{c,[b,c]}]

composite(Function1, Function2)

-spec composite(Function1, Function2) -> Function3
                   when Function1 :: a_function(), Function2 :: a_function(), Function3 :: a_function().

Returns the composite of the functions Function1 and Function2.

Examples

1> F1 = sofs:a_function([{a,1},{b,2},{c,2}]).
2> F2 = sofs:a_function([{1,x},{2,y},{3,z}]).
3> F = sofs:composite(F1, F2).
4> sofs:to_external(F).
[{a,x},{b,y},{c,y}]
5> sofs:composite(F2, F1).
** exception error: bad_function
     in function  sofs:composite/2

constant_function(Set, AnySet)

-spec constant_function(Set, AnySet) -> Function
                           when AnySet :: anyset(), Function :: a_function(), Set :: a_set().

Creates the function that maps each element of set Set onto AnySet.

Examples

1> S = sofs:set([a,b]).
2> E = sofs:from_term(1).
3> R = sofs:constant_function(S, E).
4> sofs:to_external(R).
[{a,1},{b,1}]

converse(BinRel1)

-spec converse(BinRel1) -> BinRel2 when BinRel1 :: binary_relation(), BinRel2 :: binary_relation().

Returns the converse of the binary relation BinRel1.

See inverse/1 for a similar function that applies only to invertible functions.

Examples

1> R1 = sofs:relation([{1,a},{2,b},{3,a}]).
2> R2 = sofs:converse(R1).
3> sofs:to_external(R2).
[{a,1},{a,3},{b,2}]

difference(Set1, Set2)

-spec difference(Set1, Set2) -> Set3 when Set1 :: a_set(), Set2 :: a_set(), Set3 :: a_set().

Returns the difference of the sets Set1 and Set2.

Examples

1> S0 = sofs:set([a,b,c,d]).
2> S1 = sofs:set([c,d,e,f]).
3> sofs:to_external(sofs:difference(S0, S1)).
[a,b]
4> sofs:to_external(sofs:difference(S1, S0)).
[e,f]

digraph_to_family(Graph)

-spec digraph_to_family(Graph) -> Family when Graph :: digraph:graph(), Family :: family().

Equivalent to digraph_to_family(Graph, [{atom, [atom]}]).

digraph_to_family(Graph, Type)

-spec digraph_to_family(Graph, Type) -> Family
                           when Graph :: digraph:graph(), Family :: family(), Type :: type().

Creates a family from the directed graph Graph.

Each vertex a of Graph is represented by a pair (a, {b[1], ..., b[n]}), where the b[i]:s are the out-neighbors of a. It is assumed that Type is a valid type of the external set of the family.

If G is a directed graph, it holds that the vertices and edges of G are the same as the vertices and edges of family_to_digraph(digraph_to_family(G)).

Examples

1> G = digraph:new().
2> digraph:add_vertex(G, 1).
3> digraph:add_vertex(G, a).
4> digraph:add_vertex(G, b).
5> digraph:add_edge(G, 1, a).
6> digraph:add_edge(G, 1, b).
7> F = sofs:digraph_to_family(G).
8> sofs:to_external(F).
[{1,[a,b]},{a,[]},{b,[]}]

domain(BinRel)

-spec domain(BinRel) -> Set when BinRel :: binary_relation(), Set :: a_set().

Returns the domain of the binary relation BinRel.

Examples

1> R = sofs:relation([{1,a},{1,b},{2,b},{2,c}]).
2> S = sofs:domain(R).
3> sofs:to_external(S).
[1,2]

drestriction(BinRel1, Set)

-spec drestriction(BinRel1, Set) -> BinRel2
                      when BinRel1 :: binary_relation(), BinRel2 :: binary_relation(), Set :: a_set().

Returns the difference between the binary relation BinRel1 and the restriction of BinRel1 to Set.

Examples

1> R1 = sofs:relation([{1,a},{2,b},{3,c}]).
2> S = sofs:set([2,4,6]).
3> R2 = sofs:drestriction(R1, S).
4> sofs:to_external(R2).
[{1,a},{3,c}]

drestriction(R, S) is equivalent to difference(R, restriction(R, S)).

drestriction(SetFun, Set1, Set2)

-spec drestriction(SetFun, Set1, Set2) -> Set3
                      when SetFun :: set_fun(), Set1 :: a_set(), Set2 :: a_set(), Set3 :: a_set().

Returns a subset of Set1 containing those elements that do not give an element in Set2 as the result of applying SetFun.

Examples

1> SetFun = {external, fun({_A,B,C}) -> {B,C} end}.
2> R1 = sofs:relation([{a,aa,1},{b,bb,2},{c,cc,3}]).
3> R2 = sofs:relation([{bb,2},{cc,3},{dd,4}]).
4> R3 = sofs:drestriction(SetFun, R1, R2).
5> sofs:to_external(R3).
[{a,aa,1}]

drestriction(F, S1, S2) is equivalent to difference(S1, restriction(F, S1, S2)).

empty_set()

-spec empty_set() -> Set when Set :: a_set().

Returns the untyped empty set.

empty_set/0 is equivalent to from_term([], ['_']).

Examples

1> sofs:to_external(sofs:empty_set()).
[]
2> sofs:is_empty_set(sofs:empty_set()).
true

extension(BinRel1, Set, AnySet)

-spec extension(BinRel1, Set, AnySet) -> BinRel2
                   when
                       AnySet :: anyset(),
                       BinRel1 :: binary_relation(),
                       BinRel2 :: binary_relation(),
                       Set :: a_set().

Returns the extension of BinRel1 such that for each element E in Set that does not belong to the domain of BinRel1, BinRel2 contains the pair (E, AnySet).

Examples

1> S = sofs:set([b,c]).
2> A = sofs:empty_set().
3> R = sofs:family([{a,[1,2]},{b,[3]}]).
4> X = sofs:extension(R, S, A).
5> sofs:to_external(X).
[{a,[1,2]},{b,[3]},{c,[]}]

family(Tuples)

-spec family(Tuples) -> Family when Family :: family(), Tuples :: [tuple()].

Equivalent to family(Tuples, [{atom, [atom]}]).

family(Tuples, Type)

-spec family(Tuples, Type) -> Family when Family :: family(), Tuples :: [tuple()], Type :: type().

Creates a family of subsets.

family(F, T) is equivalent to from_term(F, T) if the result is a family.

Examples

1> S = sofs:family([{1,[a,b]},{2,[c]}], [{index,[value]}]).
2> sofs:to_external(sofs:family_to_relation(S)).
[{1,a},{1,b},{2,c}]
3> S = sofs:family([{1,[a,b]},{1,[c]}], [{index,[value]}]).
** exception error: bad_function
     in function  sofs:family/2

family_difference(Family1, Family2)

-spec family_difference(Family1, Family2) -> Family3
                           when Family1 :: family(), Family2 :: family(), Family3 :: family().

If Family1 and Family2 are families, then Family3 is the family such that the index set is equal to the index set of Family1, and Family3[i] is the difference between Family1[i] and Family2[i] if Family2 maps i, otherwise Family1[i].

Examples

1> F1 = sofs:family([{a,[1,2]},{b,[3,4]}]).
2> F2 = sofs:family([{b,[4,5]},{c,[6,7]}]).
3> F3 = sofs:family_difference(F1, F2).
4> sofs:to_external(F3).
[{a,[1,2]},{b,[3]}]

family_domain(Family1)

-spec family_domain(Family1) -> Family2 when Family1 :: family(), Family2 :: family().

If Family1 is a family and Family1[i] is a binary relation for every i in the index set of Family1, then Family2 is the family with the same index set as Family1 such that Family2[i] is the domain of Family1[i].

Examples

1> FR = sofs:from_term([{a,[{1,a},{2,b},{3,c}]},{b,[]},{c,[{4,d},{5,e}]}]).
2> F = sofs:family_domain(FR).
3> sofs:to_external(F).
[{a,[1,2,3]},{b,[]},{c,[4,5]}]

family_field(Family1)

-spec family_field(Family1) -> Family2 when Family1 :: family(), Family2 :: family().

If Family1 is a family and Family1[i] is a binary relation for every i in the index set of Family1, then Family2 is the family with the same index set as Family1 such that Family2[i] is the field of Family1[i].

Examples

1> FR = sofs:from_term([{a,[{1,a},{2,b},{3,c}]},{b,[]},{c,[{4,d},{5,e}]}]).
2> F = sofs:family_field(FR).
3> sofs:to_external(F).
[{a,[1,2,3,a,b,c]},{b,[]},{c,[4,5,d,e]}]

family_field(Family1) is equivalent to family_union(family_domain(Family1), family_range(Family1)).

family_intersection(Family1)

-spec family_intersection(Family1) -> Family2 when Family1 :: family(), Family2 :: family().

If Family1 is a family and Family1[i] is a set of sets for every i in the index set of Family1, then Family2 is the family with the same index set as Family1 such that Family2[i] is the intersection of Family1[i].

If Family1[i] is an empty set for some i, a badarg exception is raised.

Examples

1> F1 = sofs:from_term([{a,[[1,2,3],[2,3,4]]},{b,[[x,y,z],[x,y]]}]).
2> F2 = sofs:family_intersection(F1).
3> sofs:to_external(F2).
[{a,[2,3]},{b,[x,y]}]
4> F3 = sofs:from_term([{a,[[1,2]]},{b,[]}]).
5> sofs:family_intersection(F3).
** exception error: bad argument
     in function  sofs:family_intersection/1

family_intersection(Family1, Family2)

-spec family_intersection(Family1, Family2) -> Family3
                             when Family1 :: family(), Family2 :: family(), Family3 :: family().

If Family1 and Family2 are families, then Family3 is the family such that the index set is the intersection of Family1:s and Family2:s index sets, and Family3[i] is the intersection of Family1[i] and Family2[i].

Examples

1> F1 = sofs:family([{a,[1,2]},{b,[3,4]},{c,[5,6]}]).
2> F2 = sofs:family([{b,[4,5]},{c,[7,8]},{d,[9,10]}]).
3> F3 = sofs:family_intersection(F1, F2).
4> sofs:to_external(F3).
[{b,[4]},{c,[]}]

family_projection(SetFun, Family1)

-spec family_projection(SetFun, Family1) -> Family2
                           when SetFun :: set_fun(), Family1 :: family(), Family2 :: family().

If Family1 is a family, then Family2 is the family with the same index set as Family1 such that Family2[i] is the result of calling SetFun with Family1[i] as argument.

Examples

1> F1 = sofs:from_term([{a,[[1,2],[2,3]]},{b,[[]]}]).
2> F2 = sofs:family_projection(fun sofs:union/1, F1).
3> sofs:to_external(F2).
[{a,[1,2,3]},{b,[]}]

family_range(Family1)

-spec family_range(Family1) -> Family2 when Family1 :: family(), Family2 :: family().

If Family1 is a family and Family1[i] is a binary relation for every i in the index set of Family1, then Family2 is the family with the same index set as Family1 such that Family2[i] is the range of Family1[i].

Examples

1> FR = sofs:from_term([{a,[{1,a},{2,b},{3,c}]},{b,[]},{c,[{4,d},{5,e}]}]).
2> F = sofs:family_range(FR).
3> sofs:to_external(F).
[{a,[a,b,c]},{b,[]},{c,[d,e]}]

family_specification(Fun, Family1)

-spec family_specification(Fun, Family1) -> Family2
                              when Fun :: spec_fun(), Family1 :: family(), Family2 :: family().

If Family1 is a family, then Family2 is the restriction of Family1 to those elements i of the index set for which Fun applied to Family1[i] returns true.

If Fun is a tuple {external, Fun2}, then Fun2 is applied to the external set of Family1[i]; otherwise Fun is applied to Family1[i].

Examples

1> F1 = sofs:family([{a,[1,2,3]},{b,[1,2]},{c,[1]}]).
2> SpecFun = fun(S) -> sofs:no_elements(S) =:= 2 end.
3> F2 = sofs:family_specification(SpecFun, F1).
4> sofs:to_external(F2).
[{b,[1,2]}]

family_to_digraph(Family)

-spec family_to_digraph(Family) -> Graph when Graph :: digraph:graph(), Family :: family().

Equivalent to family_to_digraph(Family, []).

family_to_digraph(Family, GraphType)

-spec family_to_digraph(Family, GraphType) -> Graph
                           when
                               Graph :: digraph:graph(),
                               Family :: family(),
                               GraphType :: [digraph:d_type()].

Creates a directed graph from family Family.

For each pair (a, {b[1], ..., b[n]}) of Family, vertex a and the edges (a, b[i]) for 1 <= i <= n are added to a newly created directed graph.

GraphType is passed on to digraph:new/1.

It F is a family, it holds that F is a subset of digraph_to_family(family_to_digraph(F), type(F)). Equality holds if union_of_family(F) is a subset of domain(F).

Creating a cycle in an acyclic graph exits the process with a cyclic message.

Examples

1> F1 = sofs:family([{1,[a,b]}, {2,[c,d]}, {3,[d]}, {a,[b]}]).
2> G = sofs:family_to_digraph(F1, []).
3> digraph_utils:topsort(G).
[1,a,b,2,c,3,d]
4> F2 = sofs:family([{1,[1]}]).
5> sofs:family_to_digraph(F2, [acyclic]).
** exception error: cyclic
     in function  sofs:family_to_digraph/2

family_to_relation(Family)

-spec family_to_relation(Family) -> BinRel when Family :: family(), BinRel :: binary_relation().

If Family is a family, then BinRel is the binary relation containing all pairs (i, x) such that i belongs to the index set of Family and x belongs to Family[i].

Examples

1> F = sofs:family([{a,[]}, {b,[1]}, {c,[2,3]}]).
2> R = sofs:family_to_relation(F).
3> sofs:to_external(R).
[{b,1},{c,2},{c,3}]

family_union(Family1)

-spec family_union(Family1) -> Family2 when Family1 :: family(), Family2 :: family().

If Family1 is a family and Family1[i] is a set of sets for each i in the index set of Family1, then Family2 is the family with the same index set as Family1 such that Family2[i] is the union of Family1[i].

Examples

1> F1 = sofs:from_term([{a,[[1,2],[2,3]]},{b,[[]]}]).
2> F2 = sofs:family_union(F1).
3> sofs:to_external(F2).
[{a,[1,2,3]},{b,[]}]

family_union(F) is equivalent to family_projection(fun sofs:union/1, F).

family_union(Family1, Family2)

-spec family_union(Family1, Family2) -> Family3
                      when Family1 :: family(), Family2 :: family(), Family3 :: family().

If Family1 and Family2 are families, then Family3 is the family such that the index set is the union of Family1:s and Family2:s index sets, and Family3[i] is the union of Family1[i] and Family2[i] if both map i, otherwise Family1[i] or Family2[i].

Examples

1> F1 = sofs:family([{a,[1,2]},{b,[3,4]},{c,[5,6]}]).
2> F2 = sofs:family([{b,[4,5]},{c,[7,8]},{d,[9,10]}]).
3> F3 = sofs:family_union(F1, F2).
4> sofs:to_external(F3).
[{a,[1,2]},{b,[3,4,5]},{c,[5,6,7,8]},{d,[9,10]}]

field(BinRel)

-spec field(BinRel) -> Set when BinRel :: binary_relation(), Set :: a_set().

Returns the field of the binary relation BinRel.

Examples

1> R = sofs:relation([{1,a},{1,b},{2,b},{2,c}]).
2> S = sofs:field(R).
3> sofs:to_external(S).
[1,2,a,b,c]

field(R) is equivalent to union(domain(R), range(R)).

from_external(ExternalSet, Type)

-spec from_external(ExternalSet, Type) -> AnySet
                       when ExternalSet :: external_set(), AnySet :: anyset(), Type :: type().

Creates a set from the external set ExternalSet and the type Type.

It is assumed that Type is a valid type of ExternalSet.

Examples

1> S0 = sofs:from_external([{1,[a,b]},{2,[c]}], [{x,[y]}]).
2> sofs:to_external(sofs:family_to_relation(S0)).
[{1,a},{1,b},{2,c}]
3> S1 = sofs:from_external({a,b,c}, {x,x,x}).
4> sofs:no_elements(S1).
3

from_sets(AnySet)

-spec from_sets(ListOfSets) -> Set when Set :: a_set(), ListOfSets :: [anyset()];
               (TupleOfSets) -> Ordset when Ordset :: ordset(), TupleOfSets :: tuple_of(anyset()).

Returns the unordered set containing the sets of list ListOfSets, or returns the ordered set containing the sets of the non-empty tuple TupleOfSets.

Examples

Creating an unordered set.

1> S1 = sofs:relation([{a,1},{b,2}]).
2> S2 = sofs:relation([{x,3},{y,4}]).
3> S = sofs:from_sets([S1,S2]).
4> sofs:to_external(S).
[[{a,1},{b,2}],[{x,3},{y,4}]]
5> sofs:type(S).
[[{atom,atom}]]

Creating an ordered set.

1> S1 = sofs:from_term(a).
2> S2 = sofs:from_term(b).
3> S = sofs:from_sets({S1,S2}).
4> sofs:to_external(S).
{a,b}
5> sofs:type(S).
{atom,atom}

from_term(Term)

-spec from_term(Term) -> AnySet when AnySet :: anyset(), Term :: term().

Equivalent to from_term(Term, '_').

from_term(Term, Type)

-spec from_term(Term, Type) -> AnySet when AnySet :: anyset(), Term :: term(), Type :: type().

Creates an element of Sets by traversing term Term, sorting lists, removing duplicates, and deriving or verifying a valid type for the so obtained external set.

An explicitly specified type Type can be used to limit the depth of the traversal; an atomic type stops the traversal, as shown by the following example where "foo" and {"foo"} are left unmodified:

1> S = sofs:from_term([{{"foo"},[1,1]},{"foo",[2,2]}],
                      [{atom,[atom]}]),
   sofs:to_external(S).
[{{"foo"},[1]},{"foo",[2]}]

from_term/1 can be used for creating atomic or ordered sets. The only purpose of such a set is that of later building unordered sets, as all functions in this module that do anything operate on unordered sets. Creating unordered sets from a collection of ordered sets can be the way to go if the ordered sets are big and one does not want to waste heap by rebuilding the elements of the unordered set. The following example shows that a set can be built "layer by layer":

1> A = sofs:from_term(a).
2> S = sofs:set([1,2,3]).
3> P1 = sofs:from_sets({A,S}).
4> P2 = sofs:from_term({b,[6,5,4]}).
5> Ss = sofs:from_sets([P1,P2]).
6> sofs:to_external(Ss).
[{a,[1,2,3]},{b,[4,5,6]}]

Other functions that create sets are from_external/2 and from_sets/1. Special cases of from_term/2 are a_function/1,2, empty_set/0, family/1,2, relation/1,2, and set/1,2.

image(BinRel, Set1)

-spec image(BinRel, Set1) -> Set2 when BinRel :: binary_relation(), Set1 :: a_set(), Set2 :: a_set().

Returns the image of set Set1 under the binary relation BinRel.

Examples

1> R = sofs:relation([{1,a},{2,b},{2,c},{3,d}]).
2> S1 = sofs:set([1,2]).
3> S2 = sofs:image(R, S1).
4> sofs:to_external(S2).
[a,b,c]

intersection(SetOfSets)

-spec intersection(SetOfSets) -> Set when Set :: a_set(), SetOfSets :: set_of_sets().

Returns the intersection of the set of sets SetOfSets.

Intersecting an empty set of sets exits the process with a badarg message.

Examples

1> S1 = sofs:set([a,b,c]).
2> S2 = sofs:set([b,c,d,e]).
3> S3 = sofs:set([a,b,c,d]).
4> S4 = sofs:from_sets([S1,S2,S3]).
5> S5 = sofs:intersection(S4).
6> sofs:to_external(S5).
[b,c]
7> S6 = sofs:from_sets([]).
8> sofs:intersection(S6).
** exception error: bad argument
     in function  sofs:intersection/1

intersection(Set1, Set2)

-spec intersection(Set1, Set2) -> Set3 when Set1 :: a_set(), Set2 :: a_set(), Set3 :: a_set().

Returns the intersection of Set1 and Set2.

Examples

1> S1 = sofs:set([a,b,c]).
2> S2 = sofs:set([b,c,d]).
3> S3 = sofs:intersection(S1, S2).
4> sofs:to_external(S3).
[b,c]

intersection_of_family(Family)

-spec intersection_of_family(Family) -> Set when Family :: family(), Set :: a_set().

Returns the intersection of family Family.

Intersecting an empty family exits the process with a badarg message.

Examples

1> F = sofs:family([{a,[0,2,4]},{b,[0,1,2]},{c,[2,3]}]).
2> S = sofs:intersection_of_family(F).
3> sofs:to_external(S).
[2]

inverse(Function1)

-spec inverse(Function1) -> Function2 when Function1 :: a_function(), Function2 :: a_function().

Returns the inverse of function Function1.

A bad_function exception is raised if Function1 is not invertible.

See converse/1 for a similar function that handles any binary relation.

Examples

1> F1 = sofs:relation([{1,a},{2,b},{3,c}]).
2> F2 = sofs:inverse(F1).
3> sofs:to_external(F2).
[{a,1},{b,2},{c,3}]

Trying to inverse a non-invertible function.

1> R1 = sofs:relation([{1,a},{2,a}]).
2> sofs:inverse(R1).
** exception error: bad_function
     in function  sofs:inverse/1
3> R2 = sofs:converse(R1).
4> sofs:to_external(R2).
[{a,1},{a,2}]

inverse_image(BinRel, Set1)

-spec inverse_image(BinRel, Set1) -> Set2
                       when BinRel :: binary_relation(), Set1 :: a_set(), Set2 :: a_set().

Returns the inverse image of Set1 under the binary relation BinRel.

Examples

1> R = sofs:relation([{1,a},{2,b},{2,c},{3,d}]).
2> S1 = sofs:set([c,d,e]).
3> S2 = sofs:inverse_image(R, S1).
4> sofs:to_external(S2).
[2,3]

is_a_function(BinRel)

-spec is_a_function(BinRel) -> Bool when Bool :: boolean(), BinRel :: binary_relation().

Returns true if the binary relation BinRel is a function or the untyped empty set; otherwise, returns false.

Examples

1> sofs:is_a_function(sofs:relation([{1,a},{2,b},{3,c}])).
true
2> sofs:is_a_function(sofs:relation([{1,a},{1,b},{3,c}])).
false
3> sofs:is_a_function(sofs:set([a,b,c])).
** exception error: bad argument
     in function  sofs:is_a_function/1

is_disjoint(Set1, Set2)

-spec is_disjoint(Set1, Set2) -> Bool when Bool :: boolean(), Set1 :: a_set(), Set2 :: a_set().

Returns true if Set1 and Set2 are disjoint; otherwise, returns false.

Examples

1> S1 = sofs:set([a,b,c]).
2> S2 = sofs:set([c,d,e]).
3> S3 = sofs:set([1,2,3]).
4> sofs:is_disjoint(S1, S2).
false
5> sofs:is_disjoint(S1, S3).
true
6> sofs:is_disjoint(sofs:set([1,2,3]), sofs:relation([{a,b}])).
** exception error: type_mismatch
     in function  sofs:is_disjoint/2

is_empty_set(AnySet)

-spec is_empty_set(AnySet) -> Bool when AnySet :: anyset(), Bool :: boolean().

Returns true if AnySet is an empty unordered set; otherwise, returns false.

Examples

1> sofs:is_empty_set(sofs:empty_set()).
true
2> sofs:is_empty_set(sofs:set([a,b])).
false

is_equal(AnySet1, AnySet2)

-spec is_equal(AnySet1, AnySet2) -> Bool
                  when AnySet1 :: anyset(), AnySet2 :: anyset(), Bool :: boolean().

Returns true if AnySet1 and AnySet2 are equal, otherwise false.

Examples

The following example shows that ==/2 is used when comparing sets for equality:

1> S1 = sofs:set([1.0]).
2> S2 = sofs:set([1]).
3> sofs:is_equal(S1, S2).
true

is_set(AnySet)

-spec is_set(AnySet) -> Bool when AnySet :: anyset(), Bool :: boolean().

Returns true if AnySet appears to be an unordered set, and false if AnySet is an ordered set or an atomic set or any other term.

Note that the test is shallow and this function will return true for any term that coincides with the representation of an unordered set. See also note on data types.

Examples

1> sofs:is_set(sofs:set([1,2,3])).
true
2> sofs:is_set(sofs:from_term({a,b,c})).
false
3> sofs:is_set(42).
** exception error: no function clause matching sofs:is_set(42)

is_sofs_set(Term)

-spec is_sofs_set(Term) -> Bool when Bool :: boolean(), Term :: term().

Returns true if Term appears to be an unordered set, an ordered set, or an atomic set; otherwise, returns false.

Note that this function will return true for any term that coincides with the representation of a sofs set. See also note on data types.

Examples

1> sofs:is_sofs_set(sofs:set([a,b,c])).
true
2> sofs:is_sofs_set(sofs:from_term(a)).
true
3> sofs:is_sofs_set(sofs:from_term({a,b,c})).
true
4> sofs:is_sofs_set(42).
false

is_subset(Set1, Set2)

-spec is_subset(Set1, Set2) -> Bool when Bool :: boolean(), Set1 :: a_set(), Set2 :: a_set().

Returns true if Set1 is a subset of Set2; otherwise, returns false.

1> S1 = sofs:set([2,4,6]).
2> S2 = sofs:set([1,2,3,4,5,6]).
3> sofs:is_subset(S1, S2).
true
4> sofs:is_subset(S2, S1).
false
5> sofs:is_subset(S1, S1).
true
6> S3 = sofs:relation([{1,a},{2,b}]).
7> S4 = sofs:relation([{1,a}]).
8> sofs:is_subset(S4, S3).
true
9> sofs:is_subset(S3, S1).
** exception error: type_mismatch
     in function  sofs:is_subset/2

is_type(Term)

-spec is_type(Term) -> Bool when Bool :: boolean(), Term :: term().

Returns true if term Term is a type.

Examples

1> sofs:is_type(atom).
true
2> sofs:is_type([atom]).
true
3> sofs:is_type({a,b}).
true
4> sofs:is_type(42).
false

join(Relation1, I, Relation2, J)

-spec join(Relation1, I, Relation2, J) -> Relation3
              when
                  Relation1 :: relation(),
                  Relation2 :: relation(),
                  Relation3 :: relation(),
                  I :: pos_integer(),
                  J :: pos_integer().

Returns the natural join of the relations Relation1 and Relation2 on coordinates I and J.

Examples

1> R1 = sofs:relation([{a,x,1},{b,y,2}]).
2> R2 = sofs:relation([{1,f,g},{1,h,i},{2,3,4}]).
3> J = sofs:join(R1, 3, R2, 1).
4> sofs:to_external(J).
[{a,x,1,f,g},{a,x,1,h,i},{b,y,2,3,4}]

multiple_relative_product(TupleOfBinRels, BinRel1)

-spec multiple_relative_product(TupleOfBinRels, BinRel1) -> BinRel2
                                   when
                                       TupleOfBinRels :: tuple_of(BinRel),
                                       BinRel :: binary_relation(),
                                       BinRel1 :: binary_relation(),
                                       BinRel2 :: binary_relation().

If TupleOfBinRels is a non-empty tuple {R[1], ..., R[n]} of binary relations and BinRel1 is a binary relation, then BinRel2 is the multiple relative product of the ordered set (R[i], ..., R[n]) and BinRel1.

Examples

1> Ri = sofs:relation([{a,1},{b,2},{c,3}]).
2> R = sofs:relation([{a,b},{b,c},{c,a}]).
3> MP = sofs:multiple_relative_product({Ri, Ri}, R).
4> sofs:to_external(sofs:range(MP)).
[{1,2},{2,3},{3,1}]

no_elements(ASet)

-spec no_elements(ASet) -> NoElements when ASet :: a_set() | ordset(), NoElements :: non_neg_integer().

Returns the number of elements of the ordered or unordered set ASet.

Examples

1> sofs:no_elements(sofs:set([a,b,c])).
3
2> sofs:no_elements(sofs:relation([{1,a}])).
1
3> sofs:no_elements(sofs:from_term({1,2,3,4})).
4
4> sofs:no_elements(sofs:from_term(a)).
** exception error: bad argument
     in function  sofs:no_elements/1

partition(SetOfSets)

-spec partition(SetOfSets) -> Partition when SetOfSets :: set_of_sets(), Partition :: a_set().

Returns the partition of the union of the set of sets SetOfSets such that two elements are considered equal if they belong to the same elements of SetOfSets.

Examples

1> Sets1 = sofs:from_term([[a,b,c],[d,e,f],[g,h,i]]).
2> Sets2 = sofs:from_term([[b,c,d],[e,f,g],[h,i,j]]).
3> P = sofs:partition(sofs:union(Sets1, Sets2)).
4> sofs:to_external(P).
[[a],[b,c],[d],[e,f],[g],[h,i],[j]]

partition(SetFun, Set)

-spec partition(SetFun, Set) -> Partition when SetFun :: set_fun(), Partition :: a_set(), Set :: a_set().

Returns the partition of Set such that two elements are considered equal if the results of applying SetFun are equal.

Examples

1> Ss = sofs:from_term([[a],[b],[c,d],[e,f]]).
2> SetFun = fun(S) -> sofs:from_term(sofs:no_elements(S)) end.
3> P = sofs:partition(SetFun, Ss).
4> sofs:to_external(P).
[[[a],[b]],[[c,d],[e,f]]]

partition(SetFun, Set1, Set2)

-spec partition(SetFun, Set1, Set2) -> {Set3, Set4}
                   when
                       SetFun :: set_fun(),
                       Set1 :: a_set(),
                       Set2 :: a_set(),
                       Set3 :: a_set(),
                       Set4 :: a_set().

Returns a pair of sets that, regarded as constituting a set, forms a partition of Set1.

If the result of applying SetFun to an element of Set1 gives an element in Set2, the element belongs to Set3, otherwise the element belongs to Set4.

partition(F, S1, S2) is equivalent to {restriction(F, S1, S2), drestriction(F, S1, S2)}.

Examples

1> R1 = sofs:relation([{1,a},{2,b},{3,c}]).
2> S = sofs:set([2,4,6]).
3> {R2,R3} = sofs:partition(1, R1, S).
4> {sofs:to_external(R2),sofs:to_external(R3)}.
{[{2,b}],[{1,a},{3,c}]}

partition_family(SetFun, Set)

-spec partition_family(SetFun, Set) -> Family
                          when Family :: family(), SetFun :: set_fun(), Set :: a_set().

Returns family Family where the indexed set is a partition of Set such that two elements are considered equal if the results of applying SetFun are the same value i.

This is the index that Family maps onto the equivalence class.

Examples

1> S = sofs:relation([{a,a,a,a},{a,a,b,b},{a,b,b,b}]).
2> SetFun = {external, fun({A,_,C,_}) -> {A,C} end}.
3> F = sofs:partition_family(SetFun, S).
4> sofs:to_external(F).
[{{a,a},[{a,a,a,a}]},{{a,b},[{a,a,b,b},{a,b,b,b}]}]

product(TupleOfSets)

-spec product(TupleOfSets) -> Relation when Relation :: relation(), TupleOfSets :: tuple_of(a_set()).

Returns the Cartesian product of the non-empty tuple of sets TupleOfSets.

If (x[1], ..., x[n]) is an element of the n-ary relation Relation, then x[i] is drawn from element i of TupleOfSets.

Examples

1> S1 = sofs:set([a,b]).
2> S2 = sofs:set([1,2]).
3> S3 = sofs:set([x,y]).
4> P3 = sofs:product({S1,S2,S3}).
5> sofs:to_external(P3).
[{a,1,x},{a,1,y},{a,2,x},{a,2,y},{b,1,x},{b,1,y},{b,2,x},{b,2,y}]

product(Set1, Set2)

-spec product(Set1, Set2) -> BinRel when BinRel :: binary_relation(), Set1 :: a_set(), Set2 :: a_set().

Returns the Cartesian product of Set1 and Set2.

Examples

1> S1 = sofs:set([1,2]).
2> S2 = sofs:set([a,b]).
3> R = sofs:product(S1, S2).
4> sofs:to_external(R).
[{1,a},{1,b},{2,a},{2,b}]

product(S1, S2) is equivalent to product({S1, S2}).

projection(SetFun, Set1)

-spec projection(SetFun, Set1) -> Set2 when SetFun :: set_fun(), Set1 :: a_set(), Set2 :: a_set().

Returns the set created by substituting each element of Set1 by the result of applying SetFun to the element.

If SetFun is a number i >= 1 and Set1 is a relation, then the returned set is the projection of Set1 onto coordinate i.

Examples

1> S1 = sofs:from_term([{1,a},{2,b},{3,a}]).
2> S2 = sofs:projection(2, S1).
3> sofs:to_external(S2).
[a,b]

Projecting using an external SetFun.

1> S1 = sofs:relation([{1,2,7}, {4,3,2}]).
2> SetFun = {external,fun({X,_,Z}) -> {X,Z} end}.
3> S2 = sofs:projection(SetFun, S1).
4> sofs:to_external(S2).
[{1,7},{4,2}]

range(BinRel)

-spec range(BinRel) -> Set when BinRel :: binary_relation(), Set :: a_set().

Returns the range of the binary relation BinRel.

Examples

1> R = sofs:relation([{1,a},{1,b},{2,b},{2,c}]).
2> S = sofs:range(R).
3> sofs:to_external(S).
[a,b,c]

relation(Tuples)

-spec relation(Tuples) -> Relation when Relation :: relation(), Tuples :: [tuple()].

Equivalent to relation(Tuples, Type), where Type is the size of the first tuple of Tuples, if such a tuple exists.

If tuples is [], then Type is 2.

Examples

1> S1 = sofs:relation([{1,a},{1,b},{1,a}]).
2> sofs:to_external(S1).
[{1,a},{1,b}]
3> sofs:type(S1).
[{atom,atom}]
4> sofs:type(sofs:relation([])).
[{atom,atom}]
5> sofs:type(sofs:relation([], 3)).
[{atom,atom,atom}]
6> sofs:relation([a,b,c]).
** exception error: bad argument
     in function  sofs:relation/1

relation(Tuples, Type)

-spec relation(Tuples, Type) -> Relation
                  when N :: integer(), Type :: N | type(), Relation :: relation(), Tuples :: [tuple()].

Creates a relation.

relation(R, T) is equivalent to from_term(R, T), if T is a type and the result is a relation.

If Type is an integer N, then [{atom, ..., atom}]), where the tuple size is N, is used as type of the relation.

Examples

1> S1 = sofs:relation([{3,blue},{2,green},{3,blue},{1,red}], [{index,color}]).
2> sofs:to_external(S1).
[{1,red},{2,green},{3,blue}]
3> sofs:type(S1).
[{index,color}]
4> sofs:type(sofs:relation([{1,a},{1,b}], 2)).
[{atom,atom}]
5> sofs:type(sofs:relation([], 3)).
[{atom,atom,atom}]

relation_to_family(BinRel)

-spec relation_to_family(BinRel) -> Family when Family :: family(), BinRel :: binary_relation().

Returns family Family such that the index set is equal to the domain of the binary relation BinRel, and Family[i] is the image of the set of i under BinRel.

Examples

1> R = sofs:relation([{b,1},{c,2},{c,3}]).
2> F = sofs:relation_to_family(R).
3> sofs:to_external(F).
[{b,[1]},{c,[2,3]}]

relative_product1(BinRel1, BinRel2)

-spec relative_product1(BinRel1, BinRel2) -> BinRel3
                           when
                               BinRel1 :: binary_relation(),
                               BinRel2 :: binary_relation(),
                               BinRel3 :: binary_relation().

Returns the relative product of the converse of the binary relation BinRel1 and the binary relation BinRel2.

Examples

1> R1 = sofs:relation([{1,a},{1,aa},{2,b}]).
2> R2 = sofs:relation([{1,u},{2,v},{3,c}]).
3> R3 = sofs:relative_product1(R1, R2).
4> sofs:to_external(R3).
[{a,u},{aa,u},{b,v}]

relative_product1(R1, R2) is equivalent to relative_product(converse(R1), R2).

relative_product(ListOfBinRels)

-spec relative_product(ListOfBinRels) -> BinRel2
                          when
                              ListOfBinRels :: [BinRel, ...],
                              BinRel :: binary_relation(),
                              BinRel2 :: binary_relation().

Returns relative product of the ordered set (R[i], ..., R[n]) and the relation of equality between the elements of the Cartesian product of the ranges of R[i], range R[1] × ... × range R[n].

Examples

1> TR = sofs:relation([{1,a},{1,aa},{2,b},{4,x}]).
2> R1 = sofs:relation([{1,u},{2,v},{3,c}]).
3> R2 = sofs:relative_product([TR, R1]).
4> sofs:to_external(R2).
[{1,{a,u}},{1,{aa,u}},{2,{b,v}}]

relative_product(ListOrRel, BinRel1)

-spec relative_product(ListOfBinRels, BinRel1) -> BinRel2
                          when
                              ListOfBinRels :: [BinRel, ...],
                              BinRel :: binary_relation(),
                              BinRel1 :: binary_relation(),
                              BinRel2 :: binary_relation();
                      (BinRel1, BinRel2) -> BinRel3
                          when
                              BinRel1 :: binary_relation(),
                              BinRel2 :: binary_relation(),
                              BinRel3 :: binary_relation().

Returns the relative product.

If ListOrRel is a non-empty list [R[1], ..., R[n]] of binary relations and BinRel1 is a binary relation, then BinRel2 is the relative product of the ordered set (R[i], ..., R[n]) and BinRel1.

Notice that relative_product([R1], R2) is different from relative_product(R1, R2); the list of one element is not identified with the element itself.

Examples

1> R1 = sofs:relation([{a,b},{c,a}]).
2> R2 = sofs:relation([{a,1},{a,2}]).
3> S = sofs:from_term([{{b,1},b1},{{b,2},b2}]).
4> R3 = sofs:relative_product([R1,R2], S).
5> sofs:to_external(R3).
[{a,b1},{a,b2}]

If ListOrRel is a binary relation, then BinRel2 is the relative product of the binary relations ListOfRel and BinRel1.

Examples

1> R1 = sofs:relation([{a,b}, {c,a}]).
2> R2 = sofs:relation([{a,1}, {a,2}]).
3> R3 = sofs:relative_product(R1, R2).
4> sofs:to_external(R3).
[{c,1},{c,2}]

restriction(BinRel1, Set)

-spec restriction(BinRel1, Set) -> BinRel2
                     when BinRel1 :: binary_relation(), BinRel2 :: binary_relation(), Set :: a_set().

Returns the restriction of the binary relation BinRel1 to Set.

Examples

1> R1 = sofs:relation([{1,a},{2,b},{3,c}]).
2> S = sofs:set([1,2,4]).
3> R2 = sofs:restriction(R1, S).
4> sofs:to_external(R2).
[{1,a},{2,b}]

restriction(SetFun, Set1, Set2)

-spec restriction(SetFun, Set1, Set2) -> Set3
                     when SetFun :: set_fun(), Set1 :: a_set(), Set2 :: a_set(), Set3 :: a_set().

Returns a subset of Set1 containing those elements that gives an element in Set2 as the result of applying SetFun.

Examples

1> S1 = sofs:relation([{1,a},{2,b},{3,c}]).
2> S2 = sofs:set([b,c,d]).
3> S3 = sofs:restriction(2, S1, S2).
4> sofs:to_external(S3).
[{2,b},{3,c}]

set(Terms)

-spec set(Terms) -> Set when Set :: a_set(), Terms :: [term()].

Equivalent to set(Terms, [atom]).

set(Terms, Type)

-spec set(Terms, Type) -> Set when Set :: a_set(), Terms :: [term()], Type :: type().

Creates an unordered set.

set(L, T) is equivalent to from_term(L, T) if the result is an unordered set.

Examples

1> S1 = sofs:set([3,1,2,3,4], [digit]).
2> sofs:to_external(S1).
[1,2,3,4]
3> S2 = sofs:from_term([1,2,3,4], [digit]).
4> sofs:is_equal(S1, S2).
true

specification(Fun, Set1)

-spec specification(Fun, Set1) -> Set2 when Fun :: spec_fun(), Set1 :: a_set(), Set2 :: a_set().

Returns the set containing every element of Set1 for which Fun returns true.

If Fun is a tuple {external, Fun2}, Fun2 is applied to the external set of each element, otherwise Fun is applied to each element.

Examples

1> R1 = sofs:relation([{a,1},{b,2}]).
2> R2 = sofs:relation([{x,1},{x,2},{y,3}]).
3> S1 = sofs:from_sets([R1,R2]).
4> S2 = sofs:specification(fun sofs:is_a_function/1, S1).
5> sofs:to_external(S2).
[[{a,1},{b,2}]]

Using an external fun.

1> S1 = sofs:set([1,2,3,4,5,6,7]).
2> SetFun = {external,fun(E) -> E rem 2 =:= 0 end}.
3> S2 = sofs:specification(SetFun, S1).
4> sofs:to_external(S2).
[2,4,6]

strict_relation(BinRel1)

-spec strict_relation(BinRel1) -> BinRel2
                         when BinRel1 :: binary_relation(), BinRel2 :: binary_relation().

Returns the strict relation corresponding to the binary relation BinRel1.

Examples

1> R1 = sofs:relation([{1,1},{1,2},{2,1},{2,2}]).
2> R2 = sofs:strict_relation(R1).
3> sofs:to_external(R2).
[{1,2},{2,1}]

substitution(SetFun, Set1)

-spec substitution(SetFun, Set1) -> Set2 when SetFun :: set_fun(), Set1 :: a_set(), Set2 :: a_set().

Returns a function with the domain Set1, where each element maps to the result of applying SetFun to it.

Examples

1> R = sofs:relation([{a,1},{b,2}]).
2> sofs:to_external(sofs:projection(1, R)).
[a,b]
3> sofs:to_external(sofs:substitution(1, R)).
[{{a,1},a},{{b,2},b}]
4> SetFun = {external, fun({A,_}=E) -> {E,A} end}.
5> sofs:to_external(sofs:projection(SetFun, R)).
[{{a,1},a},{{b,2},b}]

The relation of equality between the elements of {a,b,c}:

1> I = sofs:substitution(fun(A) -> A end, sofs:set([a,b,c])).
2> sofs:to_external(I).
[{a,a},{b,b},{c,c}]

Let SetOfSets be a set of sets and BinRel a binary relation. The function that maps each element Set of SetOfSets onto the image of Set under BinRel is returned by the Images fun in the following example.

1> Images = fun(SetOfSets, BinRel) ->
                    Fun = fun(Set) -> sofs:image(BinRel, Set) end,
                    sofs:substitution(Fun, SetOfSets)
            end.
2> S1 = sofs:set([1,2]).
3> S2 = sofs:set([1,3,4]).
4> S3 = sofs:set([x]).
5> SetsOfSets = sofs:from_sets([S1,S2,S3]).
6> BinRel = sofs:relation([{1,a}, {2,b}, {3,c}, {4,d}]).
7> S4 = Images(SetsOfSets, BinRel).
8> sofs:to_external(S4).
[{[1,2],[a,b]},{[1,3,4],[a,c,d]},{[x],[]}]

External unordered sets are represented as sorted lists. So, creating the image of a set under a relation R can traverse all elements of R (to that comes the sorting of results, the image). In the Image fun, BinRel is traversed once for each element of SetOfSets.

The following Images2 fun is more efficient. It can can be used under the assumption that the image of each element of SetOfSets under BinRel is non-empty.

1> Images2 = fun(SetOfSets, BinRel) ->
                 CR = sofs:canonical_relation(SetOfSets),
                 R = sofs:relative_product1(CR, BinRel),
                 sofs:relation_to_family(R)
   end.
2> S1 = sofs:set([1,2]).
3> S2 = sofs:set([1,3,4]).
4> S3 = sofs:set([x]).
5> SetsOfSets = sofs:from_sets([S1,S2,S3]).
6> BinRel = sofs:relation([{1,a}, {2,b}, {3,c}, {4,d}]).
7> S4 = Images2(SetsOfSets, BinRel).
8> sofs:to_external(S4).
[{[1,2],[a,b]},{[1,3,4],[a,c,d]}]

Note that S3, which has an empty image, is missing from the result.

symdiff(Set1, Set2)

-spec symdiff(Set1, Set2) -> Set3 when Set1 :: a_set(), Set2 :: a_set(), Set3 :: a_set().

Returns the symmetric difference (or the Boolean sum) of Set1 and Set2.

Examples

1> S1 = sofs:set([1,2,3]).
2> S2 = sofs:set([2,3,4]).
3> P = sofs:symdiff(S1, S2).
4> sofs:to_external(P).
[1,4]

symmetric_partition(Set1, Set2)

-spec symmetric_partition(Set1, Set2) -> {Set3, Set4, Set5}
                             when
                                 Set1 :: a_set(),
                                 Set2 :: a_set(),
                                 Set3 :: a_set(),
                                 Set4 :: a_set(),
                                 Set5 :: a_set().

Returns the symmetric partition of Set1 and Set2.

Returns a triple of sets:

Set3 contains the elements of Set1 that do not belong to Set2.

Set4 contains the elements of Set1 that belong to Set2.

Set5 contains the elements of Set2 that do not belong to Set1.

Examples

1> S1 = sofs:set([a,b,c]).
2> S2 = sofs:set([c,d,e]).
3> {S3,S4,S5} = sofs:symmetric_partition(S1, S2).
4> {sofs:to_external(S3),sofs:to_external(S4),sofs:to_external(S5)}
{[a,b],[c],[d,e]}

to_external(AnySet)

-spec to_external(AnySet) -> ExternalSet when ExternalSet :: external_set(), AnySet :: anyset().

Returns the external set of an atomic, ordered, or unordered set.

1> sofs:to_external(sofs:set([2,3,1])).
[1,2,3]
2> sofs:to_external(sofs:from_term({2,3,1})).
{2,3,1}
3> sofs:to_external(sofs:from_term(a)).
a

to_sets(ASet)

-spec to_sets(ASet) -> Sets
                 when
                     ASet :: a_set() | ordset(), Sets :: tuple_of(AnySet) | [AnySet], AnySet :: anyset().

Returns the elements of the ordered set ASet as a tuple of sets, and the elements of the unordered set ASet as a sorted list of sets without duplicates.

Examples

1> [S1,S2,S3] = sofs:to_sets(sofs:set([3,2,1])).
2> {sofs:to_external(S1),sofs:to_external(S2),sofs:to_external(S3)}.
{1,2,3}
3> {S4,S5,S6} = sofs:to_sets(sofs:from_term({c,a,b})).
4> {sofs:to_external(S4),sofs:to_external(S5),sofs:to_external(S6)}.
{c,a,b}

type(AnySet)

-spec type(AnySet) -> Type when AnySet :: anyset(), Type :: type().

Returns the type of an atomic, ordered, or unordered set.

Examples

Unordered sets.

1> sofs:type(sofs:empty_set()).
['_']
2> sofs:type(sofs:set([], [color])).
[color]
3> sofs:type(sofs:set([red,green,blue], [color])).
[color]
4> sofs:type(sofs:set([1,2,3])).
[atom]

Ordered sets.

1> sofs:type(sofs:from_term({a,b,c})).
{atom,atom,atom}
2> sofs:type(sofs:from_term({1.0,2.5,-1.0}, {x,y,z})).
{x,y,z}

Atomic sets.

1> sofs:type(sofs:from_term(a)).
atom
2> sofs:type(sofs:from_term(1, index)).
index

union(SetOfSets)

-spec union(SetOfSets) -> Set when Set :: a_set(), SetOfSets :: set_of_sets().

Returns the union of the set of sets SetOfSets.

Examples

1> S1 = sofs:set([a,b,c]).
2> S2 = sofs:set([b,1,2]).
3> S3 = sofs:set([a,d,e])
4> S4 = sofs:from_sets([S1,S2,S3]).
5> S5 = sofs:union(S4).
6> sofs:to_external(S5).
[1,2,a,b,c,d,e]

union(Set1, Set2)

-spec union(Set1, Set2) -> Set3 when Set1 :: a_set(), Set2 :: a_set(), Set3 :: a_set().

Returns the union of Set1 and Set2.

Examples

1> S1 = sofs:set([a,b,c]).
2> S2 = sofs:set([c,d,1,2,3]).
3> S3 = sofs:union(S1, S2).
4> sofs:to_external(S3).
[1,2,3,a,b,c,d]

union_of_family(Family)

-spec union_of_family(Family) -> Set when Family :: family(), Set :: a_set().

Returns the union of family Family.

Examples

1> F = sofs:family([{a,[0,2,4]},{b,[0,1,2]},{c,[2,3]}]).
2> S = sofs:union_of_family(F).
3> sofs:to_external(S).
[0,1,2,3,4]

weak_relation(BinRel1)

-spec weak_relation(BinRel1) -> BinRel2 when BinRel1 :: binary_relation(), BinRel2 :: binary_relation().

Returns a subset S of the weak relation W corresponding to the binary relation BinRel1.

Let F be the field of BinRel1. The subset S is defined so that x S y if x W y for some x in F and for some y in F.

Examples

1> R1 = sofs:relation([{1,1},{1,2},{3,1}]).
2> R2 = sofs:weak_relation(R1).
3> sofs:to_external(R2).
[{1,1},{1,2},{2,2},{3,1},{3,3}]

binary

Library for handling binary data.

This module provides functions for manipulating byte-oriented binaries. While most of these functions could be implemented using the bit syntax, the functions in this module are highly optimized and are expected to either execute faster or consume less memory, or both, compared to equivalent implementations written in pure Erlang.

The module is provided according to EEP 31: Binary manipulation and searching module.

Note

This module handles byte-oriented data. For bitstrings that are not binaries (does not contain whole octets of bits) a badarg exception is raised from any of the functions in this module.

Summary

Types

cp()

Opaque data type representing a compiled search pattern.

part()

A representation of a part (or range) in a binary. Start is a zero-based offset into a binary/0 and Length is the length of that part.

Functions

at(Subject, Pos)

Returns the byte at position Pos (zero-based) in binary Subject as an integer.

bin_to_list(Subject)

Converts Subject to a list of byte()s, each representing the value of one byte.

bin_to_list(Subject, PosLen)

Equivalent to bin_to_list(Subject, Pos, Len).

bin_to_list(Subject, Pos, Len)

Converts part of Subject to a list of byte/0s, each representing the value of one byte.

compile_pattern(Pattern)

Builds an internal structure representing a compilation of a search pattern, later to be used in functions match/3, matches/3, split/3, or replace/4.

copy(Subject)

Creates a copy of Subject.

copy(Subject, N)

Creates a binary with the content of Subject duplicated N times.

decode_hex(Bin)

Decodes a hex-encoded binary into a binary.

decode_unsigned(Subject)

Equivalent to decode_unsigned(Subject, big).

decode_unsigned(Subject, Endianness)

Converts the binary digit representation, in big endian or little endian, of a positive integer in Subject to an Erlang integer/0.

encode_hex(Bin)

Equivalent to encode_hex(Bin, uppercase).

encode_hex(Bin, Case)

Encodes a binary into a hex-encoded binary using the specified case for the hexadecimal digits "a" to "f".

encode_unsigned(Unsigned)

Equivalent to encode_unsigned(Unsigned, big).

encode_unsigned(Unsigned, Endianness)

Converts a non-negative integer into the smallest possible unsigned binary representation, using either big-endian or little-endian format.

first(Subject)

Returns the first byte of binary Subject as an integer.

join/2

Joins a list of binaries together by a specified Separator.

last(Subject)

Returns the last byte of binary Subject as an integer.

list_to_bin(ByteList)

Works exactly as erlang:list_to_binary/1.

longest_common_prefix(Binaries)

Returns the length of the longest common prefix of the binaries in list Binaries.

longest_common_suffix(Binaries)

Returns the length of the longest common suffix of the binaries in list Binaries.

match(Subject, Pattern)

Equivalent to match(Subject, Pattern, []).

match(Subject, Pattern, Options)

Searches for the first occurrence of Pattern in Subject and returns the position and length.

matches(Subject, Pattern)

Equivalent to matches(Subject, Pattern, []).

matches(Subject, Pattern, Options)

As match/2, but Subject is searched until exhausted and a list of all non-overlapping parts matching Pattern is returned (in order).

part(Subject, PosLen)

Equivalent to part(Subject, Pos, Len).

part(Subject, Pos, Len)

Extracts the part of binary Subject described by PosLen.

referenced_byte_size(Binary)

Get the size of the underlying binary referenced by Binary.

replace(Subject, Pattern, Replacement)

Equivalent to replace(Subject, Pattern, Replacement, []).

replace(Subject, Pattern, Replacement, Options)

Constructs a new binary by replacing the parts in Subject matching Pattern with Replacement if given as a literal binary/0 or with the result of applying Replacement to a matching subpart if given as a fun.

split(Subject, Pattern)

Equivalent to split(Subject, Pattern, []).

split(Subject, Pattern, Options)

Splits Subject into a list of binaries based on Pattern.

Types

cp()

(since OTP R14B)

-opaque cp()

Opaque data type representing a compiled search pattern.

Guaranteed to be a tuple/0 to allow programs to distinguish it from non-precompiled search patterns.

part()

(since OTP R14B)

-type part() :: {Start :: non_neg_integer(), Length :: integer()}.

A representation of a part (or range) in a binary. Start is a zero-based offset into a binary/0 and Length is the length of that part.

As input to functions in this module, a reverse part specification is allowed, constructed with a negative Length, so that the part of the binary begins at Start + Length and is -Length long. This is useful for referencing the last N bytes of a binary as {size(Binary), -N}. The functions in this module always return part/0s with positive Length.

Functions

at(Subject, Pos)

(since OTP R14B)

-spec at(Subject, Pos) -> byte() when Subject :: binary(), Pos :: non_neg_integer().

Returns the byte at position Pos (zero-based) in binary Subject as an integer.

If Pos >= byte_size(Subject), a badarg exception is raised.

Examples

1> binary:at(<<5,19,72,33>>, 0).
5
2> binary:at(<<5,19,72,33>>, 1).
19
3> binary:at(<<5,19,72,33>>, 4).
** exception error: bad argument
     in function  binary:at/2
        called as binary:at(<<5,19,72,33>>,4)

bin_to_list(Subject)

(since OTP R14B)

-spec bin_to_list(Subject) -> [byte()] when Subject :: binary().

Converts Subject to a list of byte()s, each representing the value of one byte.

Examples

1> binary:bin_to_list(<<"erlang",0>>).
[101,114,108,97,110,103,0]

bin_to_list(Subject, PosLen)

(since OTP R14B)

-spec bin_to_list(Subject, PosLen) -> [byte()] when Subject :: binary(), PosLen :: part().

Equivalent to bin_to_list(Subject, Pos, Len).

bin_to_list(Subject, Pos, Len)

(since OTP R14B)

-spec bin_to_list(Subject, Pos, Len) -> [byte()]
                     when Subject :: binary(), Pos :: non_neg_integer(), Len :: integer().

Converts part of Subject to a list of byte/0s, each representing the value of one byte.

Pos and Len denote which part of the Subject binary to convert.

Examples

1> binary:bin_to_list(<<"erlang">>, 1, 3).
"rla"
%% or [114,108,97] in list notation.
2> binary:bin_to_list(<<"erlang">>, 5, 3).
** exception error: bad argument
     in function  binary:bin_to_list/3
        called as binary:bin_to_list(<<"erlang">>,5,3)
        *** argument 3: out of range

If Pos and Len reference outside the binary in any way, a badarg exception is raised.

compile_pattern(Pattern)

(since OTP R14B)

-spec compile_pattern(Pattern) -> cp()
                         when
                             Pattern :: PatternBinary | [PatternBinary, ...],
                             PatternBinary :: nonempty_binary().

Builds an internal structure representing a compilation of a search pattern, later to be used in functions match/3, matches/3, split/3, or replace/4.

The cp/0 returned is guaranteed to be a tuple/0 to allow programs to distinguish it from non-precompiled search patterns.

When a list of binaries is specified, it denotes a set of alternative binaries to search for. For example, if [<<"functional">>,<<"programming">>] is specified as Pattern, this means either <<"functional">> or <<"programming">>". The pattern is a set of alternatives; when only a single binary is specified, the set has only one element. The order of alternatives in a pattern is not significant.

The list of binaries used for search alternatives must be flat, proper, and non-empty.

If Pattern is not a binary or a flat proper non-empty list of binaries with length greater than 0, a badarg exception is raised.

Examples

1> Pat = binary:compile_pattern(~"rain").
2> binary:match(~"the rain in spain", Pat).
{4,4}

copy(Subject)

(since OTP R14B)

-spec copy(Subject) -> binary() when Subject :: binary().

Creates a copy of Subject.

Using copy/1 on a binary that references a larger binary can potentially free up the larger binary for garbage collection.

Note

Deliberately copying a single binary to avoid referencing a larger binary does not necessarily free up the larger binary for garbage collection. Instead, it can lead to the creation of significantly more binary data than needed. In general, sharing binary data is beneficial.

Only in special cases — when small parts reference large binaries and the large binaries are no longer used in any process — can deliberate copying be beneficial.

Examples

1> HugeBinary = <<0:100_000/unit:8>>.
2> byte_size(HugeBinary).
100000
3> Part = binary:part(HugeBinary, 0, 5).
<<0,0,0,0,0>>
4> Copy = binary:copy(Part).
<<0,0,0,0,0>>

copy(Subject, N)

(since OTP R14B)

-spec copy(Subject, N) -> binary() when Subject :: binary(), N :: non_neg_integer().

Creates a binary with the content of Subject duplicated N times.

This function always creates a new binary, even when N is 1.

Examples

1> binary:copy(~"-", 10).
<<"----------">>

decode_hex(Bin)

(since OTP 24.0)

-spec decode_hex(Bin) -> Bin2 when Bin :: <<_:_*16>>, Bin2 :: binary().

Decodes a hex-encoded binary into a binary.

An exception is raised if the size of the binary is not evenly divisble by two, or if the binary contains any characters that do not represent hex digits.

Examples

1> binary:decode_hex(<<"666f6f">>).
<<"foo">>
2> binary:decode_hex(<<"A">>).
** exception error: bad argument
     in function  binary:decode_hex/1
        called as binary:decode_hex(<<"A">>)
        *** argument 1: must contain an even number of bytes

decode_unsigned(Subject)

(since OTP R14B)

-spec decode_unsigned(Subject) -> Unsigned when Subject :: binary(), Unsigned :: non_neg_integer().

Equivalent to decode_unsigned(Subject, big).

decode_unsigned(Subject, Endianness)

(since OTP R14B)

-spec decode_unsigned(Subject, Endianness) -> Unsigned
                         when
                             Subject :: binary(),
                             Endianness :: big | little,
                             Unsigned :: non_neg_integer().

Converts the binary digit representation, in big endian or little endian, of a positive integer in Subject to an Erlang integer/0.

Examples

1> binary:decode_unsigned(<<7>>).
7
2> binary:decode_unsigned(<<1,0>>).
256
3> binary:decode_unsigned(<<169,138,199>>).
11111111
4> binary:decode_unsigned(<<169,138,199>>, big).
11111111
5> binary:decode_unsigned(<<169,138,199>>, little).
13077161

encode_hex(Bin)

(since OTP 24.0)

-spec encode_hex(Bin) -> Bin2 when Bin :: binary(), Bin2 :: <<_:_*16>>.

Equivalent to encode_hex(Bin, uppercase).

encode_hex(Bin, Case)

(since OTP 26.0)

-spec encode_hex(Bin, Case) -> Bin2
                    when Bin :: binary(), Case :: lowercase | uppercase, Bin2 :: <<_:_*16>>.

Encodes a binary into a hex-encoded binary using the specified case for the hexadecimal digits "a" to "f".

Examples

1> binary:encode_hex(<<"foo">>, uppercase).
<<"666F6F">>
2> binary:encode_hex(<<"/">>, uppercase).
<<"2F">>
3> binary:encode_hex(<<"/">>, lowercase).
<<"2f">>

encode_unsigned(Unsigned)

(since OTP R14B)

-spec encode_unsigned(Unsigned) -> binary() when Unsigned :: non_neg_integer().

Equivalent to encode_unsigned(Unsigned, big).

encode_unsigned(Unsigned, Endianness)

(since OTP R14B)

-spec encode_unsigned(Unsigned, Endianness) -> binary()
                         when Unsigned :: non_neg_integer(), Endianness :: big | little.

Converts a non-negative integer into the smallest possible unsigned binary representation, using either big-endian or little-endian format.

If Unsigned is not a non-negative integer, a badarg exception is raised.

Examples

1> binary:encode_unsigned(0, big).
<<0>>
2> binary:encode_unsigned(255, big).
<<255>>
3> binary:encode_unsigned(256, big).
<<1,0>>
4> binary:encode_unsigned(256, little).
<<0,1>>
5> binary:encode_unsigned(11111111, big).
<<169,138,199>>
6> binary:encode_unsigned(11111111, little).
<<199,138,169>>

first(Subject)

(since OTP R14B)

-spec first(Subject) -> byte() when Subject :: binary().

Returns the first byte of binary Subject as an integer.

If the size of Subject is zero, a badarg exception is raised.

Examples

1> binary:first(<<42,99,100>>).
42
2> binary:first(<<>>).
** exception error: bad argument
     in function  binary:first/1
        called as binary:first(<<>>)
        *** argument 1: a zero-sized binary is not allowed

join/2

(since OTP 28.0)

-spec join([binary()], binary()) -> binary().

Joins a list of binaries together by a specified Separator.

Equivalent to iolist_to_binary(lists:join(Separator, Binaries)), but faster.

Examples

1> binary:join([<<"a">>, <<"b">>, <<"c">>], <<", ">>).
<<"a, b, c">>

last(Subject)

(since OTP R14B)

-spec last(Subject) -> byte() when Subject :: binary().

Returns the last byte of binary Subject as an integer.

If the size of Subject is zero, a badarg exception is raised.

Examples

1> binary:last(<<42,99,100>>).
100
2> binary:last(<<>>).
** exception error: bad argument
     in function  binary:last/1
        called as binary:last(<<>>)
        *** argument 1: a zero-sized binary is not allowed

list_to_bin(ByteList)

(since OTP R14B)

-spec list_to_bin(ByteList) -> binary() when ByteList :: iolist().

Works exactly as erlang:list_to_binary/1.

This function is provided for completeness.

longest_common_prefix(Binaries)

(since OTP R14B)

-spec longest_common_prefix(Binaries) -> non_neg_integer() when Binaries :: [binary(), ...].

Returns the length of the longest common prefix of the binaries in list Binaries.

If Binaries is not a flat non-empty list of binaries, a badarg exception is raised.

Examples

1> binary:longest_common_prefix([<<"erlang">>, <<"ergonomy">>]).
2
2> binary:longest_common_prefix([<<"erlang">>, <<"perl">>]).
0

longest_common_suffix(Binaries)

(since OTP R14B)

-spec longest_common_suffix(Binaries) -> non_neg_integer() when Binaries :: [binary(), ...].

Returns the length of the longest common suffix of the binaries in list Binaries.

If Binaries is not a flat non-empty list of binaries, a badarg exception is raised.

Examples

1> binary:longest_common_suffix([<<"erlang">>, <<"fang">>]).
3
2> binary:longest_common_suffix([<<"erlang">>, <<"perl">>]).
0

match(Subject, Pattern)

(since OTP R14B)

-spec match(Subject, Pattern) -> Found | nomatch
               when
                   Subject :: binary(),
                   Pattern :: PatternBinary | [PatternBinary, ...] | cp(),
                   PatternBinary :: nonempty_binary(),
                   Found :: part().

Equivalent to match(Subject, Pattern, []).

match(Subject, Pattern, Options)

(since OTP R14B)

-spec match(Subject, Pattern, Options) -> Found | nomatch
               when
                   Subject :: binary(),
                   Pattern :: PatternBinary | [PatternBinary, ...] | cp(),
                   PatternBinary :: nonempty_binary(),
                   Found :: part(),
                   Options :: [Option],
                   Option :: {scope, part()}.

Searches for the first occurrence of Pattern in Subject and returns the position and length.

The function returns {Pos, Length} for the binary in Pattern, starting at the lowest position in Subject.

Summary of the options:

{scope, {Start, Length}} - Only the specified part is searched. Return values still have offsets from the beginning of Subject. A negative Length is allowed as described in Types.

If none of the strings in Pattern is found, the atom nomatch is returned.

For a description of Pattern, see compile_pattern/1.

If {scope, {Start,Length}} is specified in the options such that Start > size of Subject, Start + Length < 0 or Start + Length > size of Subject, a badarg exception is raised.

Examples

1> binary:match(<<"abcde">>, [<<"bcde">>, <<"cd">>], []).
{1,4}

Even though <<"cd">> ends before <<"bcde">>, <<"bcde">> begins first and is therefore the first match. If two overlapping matches begin at the same position, the longest is returned.

1> binary:match(~"the rain in spain", ~"ain", []).
{5,3}

matches(Subject, Pattern)

(since OTP R14B)

-spec matches(Subject, Pattern) -> Found
                 when
                     Subject :: binary(),
                     Pattern :: PatternBinary | [PatternBinary, ...] | cp(),
                     PatternBinary :: nonempty_binary(),
                     Found :: [part()].

Equivalent to matches(Subject, Pattern, []).

matches(Subject, Pattern, Options)

(since OTP R14B)

-spec matches(Subject, Pattern, Options) -> Found
                 when
                     Subject :: binary(),
                     Pattern :: PatternBinary | [PatternBinary, ...] | cp(),
                     PatternBinary :: nonempty_binary(),
                     Found :: [part()],
                     Options :: [Option],
                     Option :: {scope, part()}.

As match/2, but Subject is searched until exhausted and a list of all non-overlapping parts matching Pattern is returned (in order).

The first and longest match is preferred to a shorter, which is illustrated by the following example:

1> binary:matches(<<"abcde">>,
                  [<<"bcde">>,<<"bc">>,<<"de">>],
                  []).
[{1,4}]

The result shows that <<"bcde">> is selected instead of the shorter match <<"bc">> (which would have resulted in one more match, <<"de">>). This corresponds to the behavior of POSIX regular expressions (and programs such as awk), but is not consistent with alternative matches in re (and Perl), where lexical ordering in the search pattern determines which string matches.

If none of the strings in a pattern is found, an empty list is returned.

For a description of Pattern, see compile_pattern/1. For a description of available options, see match/3.

If {scope, {Start,Length}} is specified in the options such that Start > size of Subject, Start + Length < 0 or Start + Length is > size of Subject, a badarg exception is raised.

Examples

1> binary:matches(~"the rain in spain", ~"ai", []).
[{5,2},{14,2}]

part(Subject, PosLen)

(since OTP R14B)

-spec part(Subject, PosLen) -> binary() when Subject :: binary(), PosLen :: part().

Equivalent to part(Subject, Pos, Len).

part(Subject, Pos, Len)

(since OTP R14B)

-spec part(Subject, Pos, Len) -> binary()
              when Subject :: binary(), Pos :: non_neg_integer(), Len :: integer().

Extracts the part of binary Subject described by PosLen.

A negative length can be used to extract bytes at the end of a binary.

Note

part/2 and part/3 are also available in the erlang module under the names binary_part/2 and binary_part/3. Those BIFs are allowed in guard tests.

If Pos and Len in any way references outside the binary, a badarg exception is raised.

Examples

1> Bin = <<1,2,3,4,5,6,7,8,9,10>>.
2> binary:part(Bin, 1, 3).
<<2,3,4>>
3> binary:part(Bin, byte_size(Bin), -5).
<<6,7,8,9,10>>

referenced_byte_size(Binary)

(since OTP R14B)

-spec referenced_byte_size(Binary) -> non_neg_integer() when Binary :: binary().

Get the size of the underlying binary referenced by Binary.

If a binary references a larger binary (often called a subbinary), it can be useful to determine the size of the referenced binary. This function can be used to decide when to trigger copy/1. Copying a binary can help eliminate the reference to the original, potentially large, binary that the smaller binary depends on.

Note

Binary data is shared among processes. If another process still references the larger binary, copying only the part used by this process will consume more memory without freeing the larger binary for garbage collection. Use these intrusive functions with extreme care, and only when a real problem has been identified.

Examples

store(Binary, GBSet) ->
    NewBin =
        case binary:referenced_byte_size(Binary) of
            Large when Large > 2 * byte_size(Binary) ->
                binary:copy(Binary);
            _ ->
                Binary
        end,
    gb_sets:insert(NewBin, GBSet).

In this example, we choose to copy the binary content before inserting it into gb_sets:set() if it references a binary more than twice the size of the data we want to retain. Naturally, different rules apply when copying in different applications.

Binary sharing occurs whenever binaries are taken apart. This is the fundamental reason why binaries are efficient; decomposition always has O(1) complexity. However, in rare circumstances this data sharing is undesirable. In such situations, this function, along with copy/1 can be useful for optimizing memory usage.

1> A = binary:copy(<<1>>, 1000).
<<1,1,1,1,1,_/binary>>
2> byte_size(A).
1000
3> binary:referenced_byte_size(A).
1000
4> <<B:10/binary, C/binary>> = A.
5> {byte_size(B), binary:referenced_byte_size(B)}.
{10,10}
6> {byte_size(C), binary:referenced_byte_size(C)}.
{990,1000}

In the above example, the small binary B was copied, while the larger binary C still references binary A.

replace(Subject, Pattern, Replacement)

(since OTP R14B)

-spec replace(Subject, Pattern, Replacement) -> Result
                 when
                     Subject :: binary(),
                     Pattern :: PatternBinary | [PatternBinary, ...] | cp(),
                     PatternBinary :: nonempty_binary(),
                     Replacement :: binary() | fun((binary()) -> binary()),
                     Result :: binary().

Equivalent to replace(Subject, Pattern, Replacement, []).

replace(Subject, Pattern, Replacement, Options)

(since OTP R14B)

-spec replace(Subject, Pattern, Replacement, Options) -> Result
                 when
                     Subject :: binary(),
                     Pattern :: PatternBinary | [PatternBinary, ...] | cp(),
                     PatternBinary :: nonempty_binary(),
                     Replacement :: binary() | fun((binary()) -> binary()),
                     Options :: [Option],
                     Option :: global | {scope, part()} | {insert_replaced, InsPos},
                     InsPos :: OnePos | [OnePos],
                     OnePos :: non_neg_integer(),
                     Result :: binary().

Constructs a new binary by replacing the parts in Subject matching Pattern with Replacement if given as a literal binary/0 or with the result of applying Replacement to a matching subpart if given as a fun.

If Replacement is given as a binary/0 and the matching subpart of Subject giving rise to the replacement is to be inserted in the result, option {insert_replaced, InsPos} inserts the matching part into Replacement at the specified position (or positions) before inserting Replacement into Subject. If Replacement is given as a fun instead, this option is ignored.

If any position specified in InsPos is greater than the size of the replacement binary, a badarg exception is raised.

Options global and {scope, part()} work as for split/3. The return type is always binary/0.

For a description of Pattern, see compile_pattern/1.

Examples

1> binary:replace(<<"abcde">>, [<<"b">>, <<"d">>], <<"X">>, []).
<<"aXcde">>

2> binary:replace(<<"abcde">>, [<<"b">>, <<"d">>], <<"X">>, [global]).
<<"aXcXe">>

3> binary:replace(<<"abcde">>, <<"b">>, <<"[]">>, [{insert_replaced, 1}]).
<<"a[b]cde">>

4> binary:replace(<<"abcde">>, [<<"b">>, <<"d">>], <<"[]">>, [global,{insert_replaced,1}]).
<<"a[b]c[d]e">>

5> binary:replace(<<"abcde">>, [<<"b">>, <<"d">>], <<"[]">>, [global,{insert_replaced,[1,1]}]).
<<"a[bb]c[dd]e">>

6> binary:replace(<<"abcde">>, [<<"b">>, <<"d">>], <<"[-]">>, [global,{insert_replaced,[1,2]}]).
<<"a[b-b]c[d-d]e">>

7> binary:replace(<<"abcde">>, [<<"b">>, <<"d">>], fun(M) -> <<$[, M/binary, $]>> end, []).
<<"a[b]cde">>

8> binary:replace(<<"abcde">>, [<<"b">>, <<"d">>], fun(M) -> <<$[, M/binary, $]>> end, [global]).
<<"a[b]c[d]e">>

split(Subject, Pattern)

(since OTP R14B)

-spec split(Subject, Pattern) -> Parts
               when
                   Subject :: binary(),
                   Pattern :: PatternBinary | [PatternBinary, ...] | cp(),
                   PatternBinary :: nonempty_binary(),
                   Parts :: [binary()].

Equivalent to split(Subject, Pattern, []).

split(Subject, Pattern, Options)

(since OTP R14B)

-spec split(Subject, Pattern, Options) -> Parts
               when
                   Subject :: binary(),
                   Pattern :: PatternBinary | [PatternBinary, ...] | cp(),
                   PatternBinary :: nonempty_binary(),
                   Options :: [Option],
                   Option :: {scope, part()} | trim | global | trim_all,
                   Parts :: [binary()].

Splits Subject into a list of binaries based on Pattern.

If option global is not specified, only the first occurrence of Pattern in Subject gives rise to a split.

The parts of Pattern found in Subject are not included in the result.

Summary of options:

{scope, part()} - Works as in match/3 and matches/3. Note that this only defines the scope of the search for matching strings; it does not cut the binary before splitting. The bytes before and after the scope are kept in the result. See the example below.

trim - Removes trailing empty parts of the result (as does trim in re:split/3).

trim_all - Removes all empty parts of the result.

global - Repeats the split until Subject is exhausted. Conceptually option global makes split work on the positions returned by matches/3, while it normally works on the position returned by match/3.

Example of the difference between a scope and taking the binary apart before splitting:

1> binary:split(<<"banana">>, [<<"a">>], [{scope,{2,3}}]).
[<<"ban">>,<<"na">>]
2> binary:split(binary:part(<<"banana">>,{2,3}), [<<"a">>], []).
[<<"n">>,<<"n">>]

The return type is always a list of binaries that are all referencing Subject. This means that the data in Subject is not copied to new binaries, and that Subject cannot be garbage collected until the results of the split are no longer referenced.

For a description of Pattern, see compile_pattern/1.

Examples

1> binary:split(~"the quick brown fox", ~" ", []).
[<<"the">>,<<"quick brown fox">>]
2> binary:split(~"the quick brown fox", ~" ", [global]).
[<<"the">>,<<"quick">>,<<"brown">>,<<"fox">>]
3> binary:split(<<1,255,4,0,0,0,2,3>>, [<<0,0,0>>,<<2>>], []).
[<<1,255,4>>, <<2,3>>]
4> binary:split(<<0,1,0,0,4,255,255,9>>, [<<0,0>>, <<255,255>>], [global]).
[<<0,1>>,<<4>>,<<9>>]

lists

List processing functions.

This module contains functions for list processing.

Unless otherwise stated, all functions assume that position numbering starts at 1. That is, the first element of a list is at position 1.

Two terms T1 and T2 compare equal if T1 == T2 evaluates to true. They match if T1 =:= T2 evaluates to true.

Whenever an ordering function F is expected as argument, it is assumed that the following properties hold of F for all x, y, and z:

If x F y and y F x, then x = y (F is antisymmetric).

If x F y and y F z, then x F z (F is transitive).

x F y or y F x (F is total).

An example of a typical ordering function is less than or equal to: =</2.

Summary

Functions

all(Pred, List)

Returns true if Pred(Elem) returns true for all elements Elem in List; otherwise, returns false.

any(Pred, List)

Returns true if Pred(Elem) returns true for at least one element Elem in List; otherwise, returns false.

append(ListOfLists)

Returns a list in which all sublists of ListOfLists have been concatenated.

append(List1, List2)

Returns a new list, List3, consisting of the elements of List1, followed by the elements of List2.

concat(Things)

Concatenates the text representation of the elements of Things.

delete(Elem, List1)

Returns a copy of List1 where the first element matching Elem is removed, if there is such an element.

droplast(List)

Drops the last element of a List.

dropwhile(Pred, List1)

Drops elements Elem from List1 while Pred(Elem) returns true, and then returns the remaining list.

duplicate(N, Elem)

Returns a list containing N copies of term Elem.

enumerate(List1)

Equivalent to enumerate(1, 1, List1).

enumerate(Index, List1)

Equivalent to enumerate(Index, 1, List1).

enumerate(Index, Step, List1)

Returns List1 with each element H replaced by a tuple of form {I, H}, where I is the position of H in List1.

filter(Pred, List1)

Returns a list of elements Elem in List1 for which Pred(Elem) returns true.

filtermap(Fun, List1)

Calls Fun(Elem) on successive elements Elem of List1 to update or remove elements from List1.

flatlength(DeepList)

Equivalent to length(flatten(DeepList)), but more efficient.

flatmap(Fun, List1)

Takes a function from As to lists of Bs, and a list of As (List1), producing a list of Bs by applying the function to each element in List1 and appending the resulting lists.

flatten(DeepList)

Returns a flattened version of DeepList.

flatten(DeepList, Tail)

Returns a flattened version of DeepList with tail Tail appended.

foldl(Fun, Acc0, List)

Calls Fun(Elem, AccIn) on successive elements A of List, starting with AccIn bound to Acc0.

foldr(Fun, Acc0, List)

Like foldl/3, but the list is traversed from right to left.

foreach(Fun, List)

Calls Fun(Elem) for each element Elem in List, ignoring the return value.

join(Sep, List1)

Inserts Sep between each element in List1.

keydelete(Key, N, TupleList1)

Returns a copy of TupleList1, where the first occurrence of a tuple whose Nth element compares equal to Key is removed, if there is such a tuple.

keyfind(Key, N, TupleList)

Searches the list of tuples TupleList for a tuple whose Nth element compares equal to Key.

keymap(Fun, N, TupleList1)

Returns a list of tuples where, for each tuple in TupleList1, the Nth element Term1 of the tuple has been replaced with the result of calling Fun(Term1).

keymember(Key, N, TupleList)

Returns true if TupleList contains a tuple whose Nth element compares equal to Key; otherwise, returns false.

keymerge(N, TupleList1, TupleList2)

Returns the sorted list formed by merging TupleList1 and TupleList2.

keyreplace(Key, N, TupleList1, NewTuple)

Returns a copy of TupleList1 where the first occurrence of a tuple T whose Nth element compares equal to Key is replaced with NewTuple, if there is such a tuple T.

keysearch(Key, N, TupleList)

Searches the list of tuples TupleList for a tuple whose Nth element compares equal to Key.

keysort(N, TupleList1)

Returns a list of the elements in TupleList1, sorted by the Nth element of each tuple.

keystore(Key, N, TupleList1, NewTuple)

Returns a copy of TupleList1 with the first tuple whose Nth element compares equal to Key replaced by NewTuple, or with [NewTuple] appended if no such tuple exists.

keytake(Key, N, TupleList1)

Searches the list of tuples TupleList1 for a tuple whose Nth element compares equal to Key, returning {value, Tuple, TupleList2} if found, where TupleList2 is a copy of TupleList1 with the first occurrence of Tuple removed.

last(List)

Returns the last element in List.

map(Fun, List1)

Takes a function from As to Bs and a list of As, producing a list of Bs by applying the function to each element in the list.

mapfoldl(Fun, Acc0, List1)

Combines the operations of map/2 and foldl/3 into one pass.

mapfoldr(Fun, Acc0, List1)

Combines the operations of map/2 and foldr/3 into one pass.

max(List)

Returns the first element of List that compares greater than or equal to all other elements of List.

member(Elem, List)

Returns true if Elem matches some element of List; otherwise, returns false.

merge3(List1, List2, List3)

Returns the sorted list formed by merging List1, List2, and List3.

merge(ListOfLists)

Returns the sorted list formed by merging all sublists of ListOfLists.

merge(List1, List2)

Returns the sorted list formed by merging List1 and List2.

merge(Fun, List1, List2)

Returns a sorted list formed by merging List1 and List2 based on Fun.

min(List)

Returns the first element of List that compares less than or equal to all other elements of List.

nth(N, List)

Returns the Nth element of List.

nthtail(N, List)

Returns the Nth tail of List, meaning the sublist of List starting at N+1 and continuing to the end of the list.

partition(Pred, List)

Partitions List into two lists: the first containing elements for which Pred(Elem) returns true, and the second containing elements for which Pred(Elem) returns false.

prefix(List1, List2)

Returns true if List1 is a prefix of List2; otherwise, returns false.

reverse(List1)

Returns a list containing the elements in List1 in reverse order.

reverse(List1, Tail)

Returns a list containing the elements of List1 in reverse order, with tail Tail appended.

search(Pred, List)

If there is a Value in List such that Pred(Value) returns true, returns {value, Value} for the first such Value; otherwise, returns false.

seq(From, To)

Equivalent to seq(From, To, 1).

seq(From, To, Incr)

Returns a sequence of integers that starts with From and contains the successive results of adding Incr to the previous element, until To is reached or passed (in the latter case, To is not an element of the sequence).

sort(List1)

Returns a list containing the sorted elements of List1.

sort(Fun, List1)

Returns a list of the elements in List1, sorted according to the ordering function Fun, where Fun(A, B) returns true if A compares less than or equal to B in the ordering; otherwise, it returns false.

split(N, List1)

Splits List1 into List2, containing the first N elements, and List3, containing the rest.

splitwith(Pred, List)

Partitions List into two lists according to Pred.

sublist(List1, Len)

Returns the sublist of List1 starting at position 1 and with no more than Len elements.

sublist(List1, Start, Len)

Returns the sublist of List1 starting at Start and with no more than Len elements.

subtract(List1, List2)

Returns a new list, List3, which is a copy of List1 with the following modification: for each element in List2, its first occurrence in List1 is removed.

suffix(List1, List2)

Returns true if List1 is a suffix of List2; otherwise, returns false.

sum(List)

Returns the sum of the elements in List.

takewhile(Pred, List1)

Takes elements Elem from List1 while Pred(Elem) returns true, returning the longest prefix in which all elements satisfy the predicate.

ukeymerge(N, TupleList1, TupleList2)

Returns the sorted list formed by merging TupleList1 and TupleList2 based on the Nth element of each tuple.

ukeysort(N, TupleList1)

Returns a sorted list of the elements in TupleList1, keeping only the first occurrence of tuples whose Nth elements compare equal.

umerge3(List1, List2, List3)

Returns the sorted list formed by merging List1, List2, and List3, while removing duplicates.

umerge(ListOfLists)

Returns a sorted list formed by merging all sublists in ListOfLists, while removing duplicates.

umerge(List1, List2)

Returns the sorted list formed by merging List1 and List2, while removing duplicates.

umerge(Fun, List1, List2)

Returns a sorted list by merging List1 and List2 using ordering function Fun, assuming both lists are pre-sorted according to Fun and contain no duplicates.

uniq(List1)

Returns a list containing the elements of List1 with duplicated elements removed (preserving the order of the elements).

uniq(Fun, List1)

Returns a list containing the elements of List1 without the elements for which Fun returned duplicate values (preserving the order of the elements).

unzip3(List1)

"Unzips" a list of three-tuples into three lists, where the first list contains the first element of each tuple, the second list contains the second element of each tuple, and the third list contains the third element of each tuple.

unzip(List1)

"Unzips" a list of two-tuples into two lists, where the first list contains the first element of each tuple, and the second list contains the second element of each tuple.

usort(List1)

Returns a sorted list of the elements of List1, keeping only the first occurrence of elements that compare equal.

usort(Fun, List1)

Returns a list containing the sorted elements of List1 where all except the first element of the elements comparing equal according to the ordering function Fun have been removed.

zip3(List1, List2, List3)

Equivalent to zip3(List1, List2, List3, fail).

zip3(List1, List2, List3, How)

"Zips" three lists into one list of three-tuples, where the first element of each tuple is taken from the first list, the second element is taken from the corresponding element in the second list, and the third element is taken from the corresponding element in the third list.

zip(List1, List2)

Equivalent to zip(List1, List2, fail).

zip(List1, List2, How)

"Zips" two lists into one list of two-tuples, where the first element of each tuple is taken from the first list and the second element is taken from the corresponding element in the second list.

zipwith3(Combine, List1, List2, List3)

Equivalent to zipwith3(Combine, List1, List2, List3, fail).

zipwith3(Combine, List1, List2, List3, How)

Combines the elements of three lists into a single list using the Combine fun.

zipwith(Combine, List1, List2)

Equivalent to zipwith(Combine, List1, List2, fail).

zipwith(Combine, List1, List2, How)

Combines the elements of two lists into a single list using the Combine fun.

Functions

all(Pred, List)

-spec all(Pred, List) -> boolean() when Pred :: fun((Elem :: T) -> boolean()), List :: [T], T :: term().

Returns true if Pred(Elem) returns true for all elements Elem in List; otherwise, returns false.

Examples

1> IsEven = fun(N) -> N rem 2 =:= 0 end.
2> lists:all(IsEven, [2,4,5]).
false
3> lists:all(IsEven, [2,4,6]).
true

any(Pred, List)

-spec any(Pred, List) -> boolean() when Pred :: fun((Elem :: T) -> boolean()), List :: [T], T :: term().

Returns true if Pred(Elem) returns true for at least one element Elem in List; otherwise, returns false.

Examples

1> IsEven = fun(N) -> N rem 2 =:= 0 end.
2> lists:any(IsEven, [3,5,7]).
false
3> lists:any(IsEven, [2,3,5,7]).
true

append(ListOfLists)

-spec append(ListOfLists) -> List1 when ListOfLists :: [List], List :: [T], List1 :: [T], T :: term().

Returns a list in which all sublists of ListOfLists have been concatenated.

Examples

1> lists:append([[1, 2, 3], [a, b], [4, 5, 6]]).
[1,2,3,a,b,4,5,6]

append(List1, List2)

-spec append(List1, List2) -> List3 when List1 :: [T], List2 :: [T], List3 :: [T], T :: term().

Returns a new list, List3, consisting of the elements of List1, followed by the elements of List2.

Examples

1> lists:append("abc", "def").
"abcdef"

lists:append(A, B) is equivalent to A ++ B.

concat(Things)

-spec concat(Things) -> string()
                when Things :: [Thing], Thing :: atom() | integer() | float() | string().

Concatenates the text representation of the elements of Things.

The elements of Things can be atoms, integers, floats, or strings.

Examples

1> lists:concat([doc, '/', file, '.', 3]).
"doc/file.3"

delete(Elem, List1)

-spec delete(Elem, List1) -> List2 when Elem :: T, List1 :: [T], List2 :: [T], T :: term().

Returns a copy of List1 where the first element matching Elem is removed, if there is such an element.

Examples

1> lists:delete(b, [a,b,c]).
[a,c]
2> lists:delete(x, [a,b,c]).
[a,b,c]

droplast(List)

(since OTP 17.0)

-spec droplast(List) -> InitList when List :: [T, ...], InitList :: [T], T :: term().

Drops the last element of a List.

The list must be non-empty; otherwise, the function raises a function_clause exception.

Examples

1> lists:droplast([1]).
[]
2> lists:droplast([1,2,3]).
[1,2]
3> lists:droplast([]).
** exception error: no function clause matching lists:droplast([])

dropwhile(Pred, List1)

-spec dropwhile(Pred, List1) -> List2
                   when Pred :: fun((Elem :: T) -> boolean()), List1 :: [T], List2 :: [T], T :: term().

Drops elements Elem from List1 while Pred(Elem) returns true, and then returns the remaining list.

Examples

1> lists:dropwhile(fun is_atom/1, [a,b,c,1,2,3,x,y,z]).
[1,2,3,x,y,z]
2> lists:dropwhile(fun is_integer/1, [a,b,c,1,2,3,x,y,z]).
[a,b,c,1,2,3,x,y,z]

duplicate(N, Elem)

-spec duplicate(N, Elem) -> List when N :: non_neg_integer(), Elem :: T, List :: [T], T :: term().

Returns a list containing N copies of term Elem.

Examples

1> lists:duplicate(5, xx).
[xx,xx,xx,xx,xx]

enumerate(List1)

(since OTP 25.0)

-spec enumerate(List1) -> List2
                   when List1 :: [T], List2 :: [{Index, T}], Index :: integer(), T :: term().

Equivalent to enumerate(1, 1, List1).

enumerate(Index, List1)

(since OTP 25.0)

-spec enumerate(Index, List1) -> List2
                   when List1 :: [T], List2 :: [{Index, T}], Index :: integer(), T :: term().

Equivalent to enumerate(Index, 1, List1).

enumerate(Index, Step, List1)

(since OTP 26.0)

-spec enumerate(Index, Step, List1) -> List2
                   when
                       List1 :: [T],
                       List2 :: [{Index, T}],
                       Index :: integer(),
                       Step :: integer(),
                       T :: term().

Returns List1 with each element H replaced by a tuple of form {I, H}, where I is the position of H in List1.

The enumeration starts with Index and increases by Step in each step.

That is, enumerate/3 behaves as if it were defined as follows:

enumerate(I, S, List) ->
  {List1, _ } = lists:mapfoldl(fun(T, Acc) -> {{Acc, T}, Acc+S} end, I, List),
  List1.

The default values for Index and Step are both 1.

Examples

1> lists:enumerate([a,b,c]).
[{1,a},{2,b},{3,c}]
2> lists:enumerate(10, [a,b,c]).
[{10,a},{11,b},{12,c}]
3> lists:enumerate(0, -2, [a,b,c]).
[{0,a},{-2,b},{-4,c}]

filter(Pred, List1)

-spec filter(Pred, List1) -> List2
                when Pred :: fun((Elem :: T) -> boolean()), List1 :: [T], List2 :: [T], T :: term().

Returns a list of elements Elem in List1 for which Pred(Elem) returns true.

Examples

1> IsEven = fun(N) -> N rem 2 =:= 0 end.
2> lists:filter(IsEven, [1,2,3,4,5]).
[2,4]

filtermap(Fun, List1)

(since OTP R16B01)

-spec filtermap(Fun, List1) -> List2
                   when
                       Fun :: fun((Elem) -> boolean() | {true, Value}),
                       List1 :: [Elem],
                       List2 :: [Elem | Value],
                       Elem :: term(),
                       Value :: term().

Calls Fun(Elem) on successive elements Elem of List1 to update or remove elements from List1.

Fun/1 must return either a Boolean or a tuple {true, Value}. The function returns the list of elements for which Fun returns a new value, with true being equivalent to {true, Elem}.

That is, filtermap behaves as if it were defined as follows:

filtermap(Fun, List1) ->
    lists:flatmap(fun(Elem) ->
                          case Fun(Elem) of
                              false -> [];
                              true -> [Elem];
                              {true,Value} -> [Value]
                          end
                  end, List1).

Examples

1> lists:filtermap(fun(X) ->
                           case X rem 2 of
                               0 -> {true, X div 2};
                               1 -> false
                           end
                   end, [1,2,3,4,5]).
[1,2]

flatlength(DeepList)

-spec flatlength(DeepList) -> non_neg_integer() when DeepList :: [term() | DeepList].

Equivalent to length(flatten(DeepList)), but more efficient.

Examples

1> lists:flatlength([a,[b,c,[d,e]],f,[[g,h,i]]]).
9
2> lists:flatlength([[[]]]).
0

flatmap(Fun, List1)

-spec flatmap(Fun, List1) -> List2
                 when Fun :: fun((A) -> [B]), List1 :: [A], List2 :: [B], A :: term(), B :: term().

Takes a function from As to lists of Bs, and a list of As (List1), producing a list of Bs by applying the function to each element in List1 and appending the resulting lists.

That is, flatmap behaves as if it were defined as follows:

flatmap(Fun, List1) ->
    lists:append(lists:map(Fun, List1)).

Examples

1> lists:flatmap(fun(X)-> [X,X] end, [a,b,c]).
[a,a,b,b,c,c]
2> F = fun(N) when is_integer(N) -> [10 * N];
          (_) -> []
       end, ok.
3> lists:flatmap(F, [1,2,a,b,c,3]).
[10,20,30]

flatten(DeepList)

-spec flatten(DeepList) -> List when DeepList :: [term() | DeepList], List :: [term()].

Returns a flattened version of DeepList.

Examples

1> lists:flatten([a,[b,c,[d,e]],f]).
[a,b,c,d,e,f]

flatten(DeepList, Tail)

-spec flatten(DeepList, Tail) -> List
                 when DeepList :: [term() | DeepList], Tail :: [term()], List :: [term()].

Returns a flattened version of DeepList with tail Tail appended.

Examples

1> lists:flatten([a,[b,c,[d,e]],f], [g,h,i]).
[a,b,c,d,e,f,g,h,i]

foldl(Fun, Acc0, List)

-spec foldl(Fun, Acc0, List) -> Acc1
               when
                   Fun :: fun((Elem :: T, AccIn) -> AccOut),
                   Acc0 :: term(),
                   Acc1 :: term(),
                   AccIn :: term(),
                   AccOut :: term(),
                   List :: [T],
                   T :: term().

Calls Fun(Elem, AccIn) on successive elements A of List, starting with AccIn bound to Acc0.

Fun/2 must return a new accumulator, which is passed to the next call. The function returns the final value of the accumulator. Acc0 is returned if the list is empty.

Examples

1> lists:foldl(fun(X, Sum) -> X + Sum end, 0, [1,2,3,4,5]).
15
2> lists:foldl(fun(X, Prod) -> X * Prod end, 1, [1,2,3,4,5]).
120

foldr(Fun, Acc0, List)

-spec foldr(Fun, Acc0, List) -> Acc1
               when
                   Fun :: fun((Elem :: T, AccIn) -> AccOut),
                   Acc0 :: term(),
                   Acc1 :: term(),
                   AccIn :: term(),
                   AccOut :: term(),
                   List :: [T],
                   T :: term().

Like foldl/3, but the list is traversed from right to left.

Examples

1> P = fun(A, AccIn) -> [A|AccIn] end.
2> lists:foldl(P, [], [1,2,3]).
[3,2,1]
3> lists:foldr(P, [], [1,2,3]).
[1,2,3]

foldl/3 is tail-recursive and is usually preferred to foldr/3.

foreach(Fun, List)

-spec foreach(Fun, List) -> ok when Fun :: fun((Elem :: T) -> term()), List :: [T], T :: term().

Calls Fun(Elem) for each element Elem in List, ignoring the return value.

This function is used for its side effects and the evaluation order is defined to be the same as the order of the elements in the list.

join(Sep, List1)

(since OTP 19.0)

-spec join(Sep, List1) -> List2 when Sep :: T, List1 :: [T], List2 :: [T], T :: term().

Inserts Sep between each element in List1.

Has no effect on an empty list or a singleton list.

Examples

1> lists:join(x, [a,b,c]).
[a,x,b,x,c]
2> lists:join(x, [a]).
[a]
3> lists:join(x, []).
[]

keydelete(Key, N, TupleList1)

-spec keydelete(Key, N, TupleList1) -> TupleList2
                   when
                       Key :: term(),
                       N :: pos_integer(),
                       TupleList1 :: [Tuple],
                       TupleList2 :: [Tuple],
                       Tuple :: tuple().

Returns a copy of TupleList1, where the first occurrence of a tuple whose Nth element compares equal to Key is removed, if there is such a tuple.

Examples

1> lists:keydelete(c, 1, [{b,1}, {c,55}, {d,75}]).
[{b,1},{d,75}]
2> lists:keydelete(unknown, 1, [{b,1}, {c,55}, {d,75}]).
[{b,1},{c,55},{d,75}]

keyfind(Key, N, TupleList)

-spec keyfind(Key, N, TupleList) -> Tuple | false
                 when Key :: term(), N :: pos_integer(), TupleList :: [Tuple], Tuple :: tuple().

Searches the list of tuples TupleList for a tuple whose Nth element compares equal to Key.

Returns Tuple if such a tuple is found; otherwise, returns false.

Examples

1> lists:keyfind(b, 1, [{a,10}, {b,20}, {c,30}]).
{b,20}
2> lists:keyfind(unknown, 1, [{a,10}, {b,20}, {c,30}]).
false

keymap(Fun, N, TupleList1)

-spec keymap(Fun, N, TupleList1) -> TupleList2
                when
                    Fun :: fun((Term1 :: term()) -> Term2 :: term()),
                    N :: pos_integer(),
                    TupleList1 :: [Tuple],
                    TupleList2 :: [Tuple],
                    Tuple :: tuple().

Returns a list of tuples where, for each tuple in TupleList1, the Nth element Term1 of the tuple has been replaced with the result of calling Fun(Term1).

Examples

1> Fun = fun(Atom) -> atom_to_list(Atom) end.
2> lists:keymap(Fun, 2, [{name,jane,22},{name,lizzie,20},{name,lydia,15}]).
[{name,"jane",22},{name,"lizzie",20},{name,"lydia",15}]

keymember(Key, N, TupleList)

-spec keymember(Key, N, TupleList) -> boolean()
                   when Key :: term(), N :: pos_integer(), TupleList :: [Tuple], Tuple :: tuple().

Returns true if TupleList contains a tuple whose Nth element compares equal to Key; otherwise, returns false.

Examples

1> lists:keymember(b, 1, [{a,10}, {b,20}, {c,30}]).
true
2> lists:keymember(unknown, 1, [{a,10}, {b,20}, {c,30}]).
false

keymerge(N, TupleList1, TupleList2)

-spec keymerge(N, TupleList1, TupleList2) -> TupleList3
                  when
                      N :: pos_integer(),
                      TupleList1 :: [T1],
                      TupleList2 :: [T2],
                      TupleList3 :: [T1 | T2],
                      T1 :: Tuple,
                      T2 :: Tuple,
                      Tuple :: tuple().

Returns the sorted list formed by merging TupleList1 and TupleList2.

The merge is performed on the Nth element of each tuple. Both TupleList1 and TupleList2 must be key-sorted before evaluating this function. When the key elements of the two tuples compare equal, the tuple from TupleList1 is picked before the tuple from TupleList2.

Examples

1> lists:keymerge(2, [{b, 50}], [{c, 20}, {a, 50}]).
[{c,20},{b,50},{a,50}]

keyreplace(Key, N, TupleList1, NewTuple)

-spec keyreplace(Key, N, TupleList1, NewTuple) -> TupleList2
                    when
                        Key :: term(),
                        N :: pos_integer(),
                        TupleList1 :: [Tuple],
                        TupleList2 :: [Tuple],
                        NewTuple :: Tuple,
                        Tuple :: tuple().

Returns a copy of TupleList1 where the first occurrence of a tuple T whose Nth element compares equal to Key is replaced with NewTuple, if there is such a tuple T.

Examples

1> lists:keyreplace(c, 1, [{b,1}, {c,55}, {d,75}], {new,tuple}).
[{b,1},{new,tuple},{d,75}]
2> lists:keyreplace(unknown, 1, [{b,1}, {c,55}, {d,75}], {new,tuple}).
[{b,1},{c,55},{d,75}]

keysearch(Key, N, TupleList)

-spec keysearch(Key, N, TupleList) -> {value, Tuple} | false
                   when Key :: term(), N :: pos_integer(), TupleList :: [Tuple], Tuple :: tuple().

Searches the list of tuples TupleList for a tuple whose Nth element compares equal to Key.

Returns {value, Tuple} if such a tuple is found; otherwise, returns false.

Note

This function is retained for backward compatibility. Function keyfind/3 is easier to use and more efficient.

keysort(N, TupleList1)

-spec keysort(N, TupleList1) -> TupleList2
                 when N :: pos_integer(), TupleList1 :: [Tuple], TupleList2 :: [Tuple], Tuple :: tuple().

Returns a list of the elements in TupleList1, sorted by the Nth element of each tuple.

The sort is stable.

Examples

1> lists:keysort(2, [{a, 99}, {b, 17}, {c, 50}, {d, 50}]).
[{b,17},{c,50},{d,50},{a,99}]

keystore(Key, N, TupleList1, NewTuple)

-spec keystore(Key, N, TupleList1, NewTuple) -> TupleList2
                  when
                      Key :: term(),
                      N :: pos_integer(),
                      TupleList1 :: [Tuple],
                      TupleList2 :: [Tuple, ...],
                      NewTuple :: Tuple,
                      Tuple :: tuple().

Returns a copy of TupleList1 with the first tuple whose Nth element compares equal to Key replaced by NewTuple, or with [NewTuple] appended if no such tuple exists.

Examples

1> lists:keystore(b, 1, [{a, 10}, {b, 23}, {c, 99}], {bb, 1}).
[{a, 10}, {bb, 1}, {c, 99}]
2> lists:keystore(z, 1, [{a, 10}, {b, 23}, {c, 99}], {z, 2}).
[{a, 10}, {b, 23}, {c, 99}, {z, 2}]

keytake(Key, N, TupleList1)

-spec keytake(Key, N, TupleList1) -> {value, Tuple, TupleList2} | false
                 when
                     Key :: term(),
                     N :: pos_integer(),
                     TupleList1 :: [tuple()],
                     TupleList2 :: [tuple()],
                     Tuple :: tuple().

Searches the list of tuples TupleList1 for a tuple whose Nth element compares equal to Key, returning {value, Tuple, TupleList2} if found, where TupleList2 is a copy of TupleList1 with the first occurrence of Tuple removed.

Otherwise, returns false if no such tuple is found.

Examples

1> lists:keytake(b, 1, [{a, 10}, {b, 23}, {c, 99}]).
{value,{b,23},[{a, 10},{c, 99}]}
2> lists:keytake(z, 1, [{a, 10}, {b, 23}, {c, 99}]).
false

last(List)

-spec last(List) -> Last when List :: [T, ...], Last :: T, T :: term().

Returns the last element in List.

The list must be non-empty; otherwise, the function raises a function_clause exception.

Examples

1> lists:last([1]).
1
2> lists:last([1,2,3]).
3
3> lists:last([]).
** exception error: no function clause matching lists:last([])

map(Fun, List1)

-spec map(Fun, List1) -> List2
             when Fun :: fun((A) -> B), List1 :: [A], List2 :: [B], A :: term(), B :: term().

Takes a function from As to Bs and a list of As, producing a list of Bs by applying the function to each element in the list.

Examples

1> lists:map(fun(N) -> N + 1 end, [1,2,3]).
[2,3,4]

mapfoldl(Fun, Acc0, List1)

-spec mapfoldl(Fun, Acc0, List1) -> {List2, Acc1}
                  when
                      Fun :: fun((A, AccIn) -> {B, AccOut}),
                      Acc0 :: term(),
                      Acc1 :: term(),
                      AccIn :: term(),
                      AccOut :: term(),
                      List1 :: [A],
                      List2 :: [B],
                      A :: term(),
                      B :: term().

Combines the operations of map/2 and foldl/3 into one pass.

Examples

Summing the elements in a list and double them at the same time:

1> lists:mapfoldl(fun(X, Sum) -> {2*X, X+Sum} end, 0, [1,2,3,4,5]).
{[2,4,6,8,10],15}

mapfoldr(Fun, Acc0, List1)

-spec mapfoldr(Fun, Acc0, List1) -> {List2, Acc1}
                  when
                      Fun :: fun((A, AccIn) -> {B, AccOut}),
                      Acc0 :: term(),
                      Acc1 :: term(),
                      AccIn :: term(),
                      AccOut :: term(),
                      List1 :: [A],
                      List2 :: [B],
                      A :: term(),
                      B :: term().

Combines the operations of map/2 and foldr/3 into one pass.

Note

Unless the order in which the elements are accumulated is important, prefer mapfoldl/3 as it is slighly more efficient.

Examples

Doubling the elements in list and producing a list of squares at the same time:

1> lists:mapfoldr(fun(X, Acc) -> {2*X, [X*X|Acc]} end, [], [1,2,3,4,5]).
{[2,4,6,8,10],[1,4,9,16,25]}

max(List)

-spec max(List) -> Max when List :: [T, ...], Max :: T, T :: term().

Returns the first element of List that compares greater than or equal to all other elements of List.

Examples

1> lists:max([17,19,7,55]).
55
2> lists:max([]).
** exception error: no function clause matching lists:max([])

member(Elem, List)

-spec member(Elem, List) -> boolean() when Elem :: T, List :: [T], T :: term().

Returns true if Elem matches some element of List; otherwise, returns false.

Examples

1> lists:member(2, [1,2,3]).
true
2> lists:member(nope, [1,2,3]).
false

merge3(List1, List2, List3)

-spec merge3(List1, List2, List3) -> List4
                when
                    List1 :: [X],
                    List2 :: [Y],
                    List3 :: [Z],
                    List4 :: [X | Y | Z],
                    X :: term(),
                    Y :: term(),
                    Z :: term().

Returns the sorted list formed by merging List1, List2, and List3.

All of List1, List2, and List3 must be sorted before evaluating this function.

When two elements compare equal, the element from List1, if there is such an element, is picked before the other element, otherwise the element from List2 is picked before the element from List3.

Examples

1> lists:merge3([a,o], [g,q], [j]).
[a,g,j,o,q]

merge(ListOfLists)

-spec merge(ListOfLists) -> List1 when ListOfLists :: [List], List :: [T], List1 :: [T], T :: term().

Returns the sorted list formed by merging all sublists of ListOfLists.

All sublists must be sorted before evaluating this function.

When two elements compare equal, the element from the sublist with the lowest position in ListOfLists is picked before the other element.

Examples

1> lists:merge([[b,l,l], [g,k,q]]).
[b,g,k,l,l,q]

merge(List1, List2)

-spec merge(List1, List2) -> List3
               when List1 :: [X], List2 :: [Y], List3 :: [X | Y], X :: term(), Y :: term().

Returns the sorted list formed by merging List1 and List2.

Both List1 and List2 must be sorted before evaluating this function.

When two elements compare equal, the element from List1 is picked before the element from List2.

Examples

1> lists:merge([a,o], [b,x]).
[a,b,o,x]

merge(Fun, List1, List2)

-spec merge(Fun, List1, List2) -> List3
               when
                   Fun :: fun((A, B) -> boolean()),
                   List1 :: [A],
                   List2 :: [B],
                   List3 :: [A | B],
                   A :: term(),
                   B :: term().

Returns a sorted list formed by merging List1 and List2 based on Fun.

Both List1 andList2 must be sorted according to the ordering function Fun before evaluating this function.

Fun(A, B) is to return true if A compares less than or equal to B in the ordering, otherwise false. When two elements compare equal, the element from List1 is picked before the element from List2.

Examples

1> F = fun(A, B) -> tuple_size(A) =< tuple_size(B) end.
2> lists:merge(F, [{x, y}, {a, b, c}], [{q, w}]).
[{x,y},{q,w},{a,b,c}]

min(List)

-spec min(List) -> Min when List :: [T, ...], Min :: T, T :: term().

Returns the first element of List that compares less than or equal to all other elements of List.

Examples

1> lists:min([17,19,7,55]).
7
2> lists:min([]).
** exception error: no function clause matching lists:min([])

nth(N, List)

-spec nth(N, List) -> Elem when N :: pos_integer(), List :: [T, ...], Elem :: T, T :: term().

Returns the Nth element of List.

Examples

1> lists:nth(3, [a, b, c, d, e]).
c

nthtail(N, List)

-spec nthtail(N, List) -> Tail when N :: non_neg_integer(), List :: [T, ...], Tail :: [T], T :: term().

Returns the Nth tail of List, meaning the sublist of List starting at N+1 and continuing to the end of the list.

Examples

1> lists:nthtail(3, [a, b, c, d, e]).
[d,e]
2> tl(tl(tl([a, b, c, d, e]))).
[d,e]
3> lists:nthtail(0, [a, b, c, d, e]).
[a,b,c,d,e]
4> lists:nthtail(5, [a, b, c, d, e]).
[]

partition(Pred, List)

-spec partition(Pred, List) -> {Satisfying, NotSatisfying}
                   when
                       Pred :: fun((Elem :: T) -> boolean()),
                       List :: [T],
                       Satisfying :: [T],
                       NotSatisfying :: [T],
                       T :: term().

Partitions List into two lists: the first containing elements for which Pred(Elem) returns true, and the second containing elements for which Pred(Elem) returns false.

Examples

1> lists:partition(fun(A) -> A rem 2 =:= 1 end, [1,2,3,4,5,6,7]).
{[1,3,5,7],[2,4,6]}
2> lists:partition(fun(A) -> is_atom(A) end, [a,b,1,c,d,2,3,4,e]).
{[a,b,c,d,e],[1,2,3,4]}

For a different way to partition a list, see splitwith/2.

prefix(List1, List2)

-spec prefix(List1, List2) -> boolean() when List1 :: [T], List2 :: [T], T :: term().

Returns true if List1 is a prefix of List2; otherwise, returns false.

A prefix of a list is the first part of the list, starting from the beginning and stopping at any point.

Examples

1> lists:prefix("abc", "abcdef").
true
2> lists:prefix("def", "abcdef").
false
3> lists:prefix([], "any list").
true
4> lists:prefix("abc", "abc").
true

reverse(List1)

-spec reverse(List1) -> List2 when List1 :: [T], List2 :: [T], T :: term().

Returns a list containing the elements in List1 in reverse order.

Examples

1> lists:reverse([1,2,3]).
[3,2,1]

reverse(List1, Tail)

-spec reverse(List1, Tail) -> List2 when List1 :: [T], Tail :: term(), List2 :: [T], T :: term().

Returns a list containing the elements of List1 in reverse order, with tail Tail appended.

Examples

1> lists:reverse([1, 2, 3, 4], [a, b, c]).
[4,3,2,1,a,b,c]

search(Pred, List)

(since OTP 21.0)

-spec search(Pred, List) -> {value, Value} | false
                when Pred :: fun((T) -> boolean()), List :: [T], Value :: T.

If there is a Value in List such that Pred(Value) returns true, returns {value, Value} for the first such Value; otherwise, returns false.

Examples

1> lists:search(fun is_atom/1, [1,2,3,a,b,c]).
{value,a}
2> lists:search(fun(#{a := V}) -> V =:= 42 end,
     [#{a => 1}, #{a => 42}, #{a => 100}]).
{value,#{a => 42}}

seq(From, To)

-spec seq(From, To) -> Seq when From :: integer(), To :: integer(), Seq :: [integer()].

Equivalent to seq(From, To, 1).

seq(From, To, Incr)

-spec seq(From, To, Incr) -> Seq
             when From :: integer(), To :: integer(), Incr :: integer(), Seq :: [integer()].

Returns a sequence of integers that starts with From and contains the successive results of adding Incr to the previous element, until To is reached or passed (in the latter case, To is not an element of the sequence).

Incr defaults to 1.

Failures:

If To < From - Incr and Incr > 0.

If To > From - Incr and Incr < 0.

If Incr =:= 0 and From =/= To.

The following equalities hold for all sequences:

length(lists:seq(From, To)) =:= To - From + 1
length(lists:seq(From, To, Incr)) =:= (To - From + Incr) div Incr

Examples

1> lists:seq(1, 10).
[1,2,3,4,5,6,7,8,9,10]
2> lists:seq(1, 20, 3).
[1,4,7,10,13,16,19]
3> lists:seq(1, 0, 1).
[]
4> lists:seq(10, 6, 4).
[]
5> lists:seq(1, 1, 0).
[1]

sort(List1)

-spec sort(List1) -> List2 when List1 :: [T], List2 :: [T], T :: term().

Returns a list containing the sorted elements of List1.

The sort is stable.

Examples

1> lists:sort([4,1,3,2]).
[1,2,3,4]
2> lists:sort([a,4,3,b,9]).
[3,4,9,a,b]

Since the sort is stable, the relative order of elements that compare equal is not changed:

1> lists:sort([1.0,1]).
[1.0,1]
2> lists:sort([1,1.0]).
[1,1.0]

sort(Fun, List1)

-spec sort(Fun, List1) -> List2
              when Fun :: fun((A :: T, B :: T) -> boolean()), List1 :: [T], List2 :: [T], T :: term().

Returns a list of the elements in List1, sorted according to the ordering function Fun, where Fun(A, B) returns true if A compares less than or equal to B in the ordering; otherwise, it returns false.

Examples

1> F = fun(A, B) -> tuple_size(A) =< tuple_size(B) end.
2> lists:sort(F, [{a, b, c}, {x, y}, {q, w}]).
[{x,y},{q,w},{a,b,c}]

split(N, List1)

-spec split(N, List1) -> {List2, List3}
               when N :: non_neg_integer(), List1 :: [T], List2 :: [T], List3 :: [T], T :: term().

Splits List1 into List2, containing the first N elements, and List3, containing the rest.

Examples

1> lists:split(3, [1,2,3,4,5,6,7]).
{[1,2,3],[4,5,6,7]}

splitwith(Pred, List)

-spec splitwith(Pred, List) -> {List1, List2}
                   when
                       Pred :: fun((T) -> boolean()),
                       List :: [T],
                       List1 :: [T],
                       List2 :: [T],
                       T :: term().

Partitions List into two lists according to Pred.

splitwith/2 behaves as if it were defined as follows:

splitwith(Pred, List) ->
    {takewhile(Pred, List), dropwhile(Pred, List)}.

Examples

1> lists:splitwith(fun(A) -> A rem 2 =:= 1 end, [1,2,3,4,5,6,7]).
{[1],[2,3,4,5,6,7]}
2> lists:splitwith(fun(A) -> is_atom(A) end, [a,b,1,c,d,2,3,4,e]).
{[a,b],[1,c,d,2,3,4,e]}

For a different way to partition a list, see partition/2.

sublist(List1, Len)

-spec sublist(List1, Len) -> List2
                 when List1 :: [T], List2 :: [T], Len :: non_neg_integer(), T :: term().

Returns the sublist of List1 starting at position 1 and with no more than Len elements.

It is not an error for Len to exceed the length of the list, in which case the whole list is returned.

Examples

1> lists:sublist([1,2,3,4,5], 2)
[1,2]
2> lists:sublist([1,2,3,4,5], 99)
[1,2,3,4,5]

sublist(List1, Start, Len)

-spec sublist(List1, Start, Len) -> List2
                 when
                     List1 :: [T],
                     List2 :: [T],
                     Start :: pos_integer(),
                     Len :: non_neg_integer(),
                     T :: term().

Returns the sublist of List1 starting at Start and with no more than Len elements.

It is not an error for Start+Len to exceed the length of the list.

Examples

1> lists:sublist([1,2,3,4], 2, 2).
[2,3]
2> lists:sublist([1,2,3,4], 2, 5).
[2,3,4]
3> lists:sublist([1,2,3,4], 5, 2).
[]

subtract(List1, List2)

-spec subtract(List1, List2) -> List3 when List1 :: [T], List2 :: [T], List3 :: [T], T :: term().

Returns a new list, List3, which is a copy of List1 with the following modification: for each element in List2, its first occurrence in List1 is removed.

Examples

1> lists:subtract("123212", "212").
"312"

lists:subtract(A, B) is equivalent to A -- B.

suffix(List1, List2)

-spec suffix(List1, List2) -> boolean() when List1 :: [T], List2 :: [T], T :: term().

Returns true if List1 is a suffix of List2; otherwise, returns false.

A suffix of a list is the last part of the list, starting from any position and going all the way to the end.

Examples

1> lists:suffix("abc", "abcdef").
false
2> lists:suffix("def", "abcdef").
true
3> lists:suffix([], "any list").
true
4> lists:suffix("abc", "abc").
true

sum(List)

-spec sum(List) -> number() when List :: [number()].

Returns the sum of the elements in List.

Examples

1> lists:sum([]).
0
2> lists:sum([1,2,3]).
6

takewhile(Pred, List1)

-spec takewhile(Pred, List1) -> List2
                   when Pred :: fun((Elem :: T) -> boolean()), List1 :: [T], List2 :: [T], T :: term().

Takes elements Elem from List1 while Pred(Elem) returns true, returning the longest prefix in which all elements satisfy the predicate.

Examples

1> lists:takewhile(fun is_atom/1, [a,b,c,1,2,3,x,y,z]).
[a,b,c]
2> lists:takewhile(fun is_integer/1, [a,b,c,1,2,3,x,y,z]).
[]

ukeymerge(N, TupleList1, TupleList2)

-spec ukeymerge(N, TupleList1, TupleList2) -> TupleList3
                   when
                       N :: pos_integer(),
                       TupleList1 :: [T1],
                       TupleList2 :: [T2],
                       TupleList3 :: [T1 | T2],
                       T1 :: Tuple,
                       T2 :: Tuple,
                       Tuple :: tuple().

Returns the sorted list formed by merging TupleList1 and TupleList2 based on the Nth element of each tuple.

Both TupleList1 and TupleList2 must be key-sorted without duplicates before evaluating this function.

When the Nth elements of two tuples compare equal, the tuple from TupleList1 is picked and the one from TupleList2 is removed.

Examples

1> lists:ukeymerge(1, [{a, 33}, {c, 15}], [{a, 59}, {d, 39}]).
[{a,33},{c,15},{d,39}]

ukeysort(N, TupleList1)

-spec ukeysort(N, TupleList1) -> TupleList2
                  when
                      N :: pos_integer(), TupleList1 :: [Tuple], TupleList2 :: [Tuple], Tuple :: tuple().

Returns a sorted list of the elements in TupleList1, keeping only the first occurrence of tuples whose Nth elements compare equal.

Sorting is performed on the Nth element of the tuples.

Examples

1> lists:ukeysort(2, [{a, 27}, {d, 23}, {e, 23}]).
[{d,23}, {a, 27}]

umerge3(List1, List2, List3)

-spec umerge3(List1, List2, List3) -> List4
                 when
                     List1 :: [X],
                     List2 :: [Y],
                     List3 :: [Z],
                     List4 :: [X | Y | Z],
                     X :: term(),
                     Y :: term(),
                     Z :: term().

Returns the sorted list formed by merging List1, List2, and List3, while removing duplicates.

All of List1, List2, and List3 must be sorted and contain no duplicates before evaluating this function.

When two elements compare equal, the element from List1 is picked if there is such an element, otherwise the element from List2 is picked, and the other is removed.

Examples

1> lists:umerge3([a,b], [a,d,e], [b,f]).
[a,b,d,e,f]

umerge(ListOfLists)

-spec umerge(ListOfLists) -> List1 when ListOfLists :: [List], List :: [T], List1 :: [T], T :: term().

Returns a sorted list formed by merging all sublists in ListOfLists, while removing duplicates.

All sublists must be sorted and contain no duplicates before evaluating this function.

When two elements compare equal, the element from the sublist with the lowest position in ListOfLists is picked and the other is removed.

Examples

1> lists:umerge([[a,b], [a,d,e]]).
[a,b,d,e]

umerge(List1, List2)

-spec umerge(List1, List2) -> List3
                when List1 :: [X], List2 :: [Y], List3 :: [X | Y], X :: term(), Y :: term().

Returns the sorted list formed by merging List1 and List2, while removing duplicates.

Both List1 and List2 must be sorted and contain no duplicates before evaluating this function.

When two elements compare equal, the element from List1 is picked and the one from List2 is removed.

Examples

1> lists:umerge([a,b], [a,d,e]).
[a,b,d,e]

umerge(Fun, List1, List2)

-spec umerge(Fun, List1, List2) -> List3
                when
                    Fun :: fun((A, B) -> boolean()),
                    List1 :: [A],
                    List2 :: [B],
                    List3 :: [A | B],
                    A :: term(),
                    B :: term().

Returns a sorted list by merging List1 and List2 using ordering function Fun, assuming both lists are pre-sorted according to Fun and contain no duplicates.

Fun(A, B) is to return true if A compares less than or equal to B in the ordering; otherwise, it should return false. When two elements compare equal, the element from List1 is picked and the one from List2 is removed.

Examples

1> F = fun(A, B) -> tuple_size(A) =< tuple_size(B) end.
2> lists:umerge(F, [{x, y}, {a, b, c}], [{q, w}, {x, y, z, w}]).
[{x,y},{a,b,c},{x,y,z,w}]

uniq(List1)

(since OTP 25.0)

-spec uniq(List1) -> List2 when List1 :: [T], List2 :: [T], T :: term().

Returns a list containing the elements of List1 with duplicated elements removed (preserving the order of the elements).

The first occurrence of each element is kept.

Examples

1> lists:uniq([3, 3, 1, 2, 1, 2, 3]).
[3,1,2]
2> lists:uniq([a, a, 1, b, 2, a, 3]).
[a, 1, b, 2, 3]

uniq(Fun, List1)

(since OTP 25.0)

-spec uniq(Fun, List1) -> List2 when Fun :: fun((T) -> any()), List1 :: [T], List2 :: [T], T :: term().

Returns a list containing the elements of List1 without the elements for which Fun returned duplicate values (preserving the order of the elements).

The first occurrence of each element is kept.

Examples

1> lists:uniq(fun({X, _}) -> X end, [{b, 2}, {a, 1}, {c, 3}, {a, 2}]).
[{b, 2}, {a, 1}, {c, 3}]

unzip3(List1)

-spec unzip3(List1) -> {List2, List3, List4}
                when
                    List1 :: [{A, B, C}],
                    List2 :: [A],
                    List3 :: [B],
                    List4 :: [C],
                    A :: term(),
                    B :: term(),
                    C :: term().

"Unzips" a list of three-tuples into three lists, where the first list contains the first element of each tuple, the second list contains the second element of each tuple, and the third list contains the third element of each tuple.

Examples

1> lists:unzip3([{a, 1, 2}, {b, 777, 999}]).
{[a,b],[1,777],[2,999]}

unzip(List1)

-spec unzip(List1) -> {List2, List3}
               when List1 :: [{A, B}], List2 :: [A], List3 :: [B], A :: term(), B :: term().

"Unzips" a list of two-tuples into two lists, where the first list contains the first element of each tuple, and the second list contains the second element of each tuple.

Examples

1> lists:unzip([{1, a}, {2, b}]).
{[1,2],[a,b]}

usort(List1)

-spec usort(List1) -> List2 when List1 :: [T], List2 :: [T], T :: term().

Returns a sorted list of the elements of List1, keeping only the first occurrence of elements that compare equal.

Examples

1> lists:usort([a,x,y,b,c,x,a]).
[a,b,c,x,y]
2> lists:usort([3,2,a,3,2,a,1,3,b,2,2,1]).
[1,2,3,a,b]
3> lists:usort([1.0,1]).
[1.0]
4> lists:usort([1,1.0]).
[1]

usort(Fun, List1)

-spec usort(Fun, List1) -> List2
               when Fun :: fun((T, T) -> boolean()), List1 :: [T], List2 :: [T], T :: term().

Returns a list containing the sorted elements of List1 where all except the first element of the elements comparing equal according to the ordering function Fun have been removed.

Fun(A, B) is to return true if A compares less than or equal to B in the ordering, otherwise false.

Examples

1> F = fun(A, B) -> tuple_size(A) =< tuple_size(B) end.
2> lists:usort(F, [{a, b, c}, {x, y}, {q, w}]).
[{x,y},{a,b,c}]

zip3(List1, List2, List3)

-spec zip3(List1, List2, List3) -> List4
              when
                  List1 :: [A],
                  List2 :: [B],
                  List3 :: [C],
                  List4 :: [{A, B, C}],
                  A :: term(),
                  B :: term(),
                  C :: term().

Equivalent to zip3(List1, List2, List3, fail).

zip3(List1, List2, List3, How)

(since OTP 26.0)

-spec zip3(List1, List2, List3, How) -> List4
              when
                  List1 :: [A],
                  List2 :: [B],
                  List3 :: [C],
                  List4 :: [{A | DefaultA, B | DefaultB, C | DefaultC}],
                  A :: term(),
                  B :: term(),
                  C :: term(),
                  How :: fail | trim | {pad, {DefaultA, DefaultB, DefaultC}},
                  DefaultA :: term(),
                  DefaultB :: term(),
                  DefaultC :: term().

"Zips" three lists into one list of three-tuples, where the first element of each tuple is taken from the first list, the second element is taken from the corresponding element in the second list, and the third element is taken from the corresponding element in the third list.

For a description of the How parameter, see zip/3.

Examples

1> lists:zip3([a], [1, 2, 3], [17, 19], trim).
[{a,1,17}]
2> lists:zip3([a], [1, 2, 3], [17, 19], {pad, {z, 0, 0}}).
[{a,1,17}, {z,2,19}, {z,3,0}]

zip(List1, List2)

-spec zip(List1, List2) -> List3
             when List1 :: [A], List2 :: [B], List3 :: [{A, B}], A :: term(), B :: term().

Equivalent to zip(List1, List2, fail).

zip(List1, List2, How)

(since OTP 26.0)

-spec zip(List1, List2, How) -> List3
             when
                 List1 :: [A],
                 List2 :: [B],
                 List3 :: [{A | DefaultA, B | DefaultB}],
                 A :: term(),
                 B :: term(),
                 How :: fail | trim | {pad, {DefaultA, DefaultB}},
                 DefaultA :: term(),
                 DefaultB :: term().

"Zips" two lists into one list of two-tuples, where the first element of each tuple is taken from the first list and the second element is taken from the corresponding element in the second list.

The How parameter specifies the behavior if the given lists are of different lengths.

fail - The call will fail if the given lists are not of equal length. This is the default.

trim - Surplus elements from the longer list will be ignored.

Examples

1> lists:zip([a, b], [1, 2, 3], trim).
[{a,1},{b,2}]
2> lists:zip([a, b, c], [1, 2], trim).
[{a,1},{b,2}]

{pad, Defaults} - The shorter list will be padded to the length of the longer list, using the respective elements from the given Defaults tuple.

Examples

1> lists:zip([a, b], [1, 2, 3], {pad, {x, 0}}).
[{a,1},{b,2},{x,3}]
2> lists:zip([a, b, c], [1, 2], {pad, {x, 0}}).
[{a,1},{b,2},{c,0}]

zipwith3(Combine, List1, List2, List3)

-spec zipwith3(Combine, List1, List2, List3) -> List4
                  when
                      Combine :: fun((X, Y, Z) -> T),
                      List1 :: [X],
                      List2 :: [Y],
                      List3 :: [Z],
                      List4 :: [T],
                      X :: term(),
                      Y :: term(),
                      Z :: term(),
                      T :: term().

Equivalent to zipwith3(Combine, List1, List2, List3, fail).

zipwith3(Combine, List1, List2, List3, How)

(since OTP 26.0)

-spec zipwith3(Combine, List1, List2, List3, How) -> List4
                  when
                      Combine :: fun((X | DefaultX, Y | DefaultY, Z | DefaultZ) -> T),
                      List1 :: [X],
                      List2 :: [Y],
                      List3 :: [Z],
                      List4 :: [T],
                      X :: term(),
                      Y :: term(),
                      Z :: term(),
                      How :: fail | trim | {pad, {DefaultX, DefaultY, DefaultZ}},
                      DefaultX :: term(),
                      DefaultY :: term(),
                      DefaultZ :: term(),
                      T :: term().

Combines the elements of three lists into a single list using the Combine fun.

For each triple X, Y, Z of list elements from the three lists, the element in the result list is Combine(X, Y, Z).

For a description of the How parameter, see zip/3.

zipwith3(fun(X, Y, Z) -> {X,Y,Z} end, List1, List2, List3) is equivalent to zip3(List1, List2, List3).

Examples

1> lists:zipwith3(fun(X, Y, Z) -> X+Y+Z end, [1,2,3], [4,5,6], [7,8,9], fail).
[12,15,18]
2> lists:zipwith3(fun(X, Y, Z) -> [X,Y,Z] end, [a,b,c], [x,y,z], [1,2,3], fail).
[[a,x,1],[b,y,2],[c,z,3]]

zipwith(Combine, List1, List2)

-spec zipwith(Combine, List1, List2) -> List3
                 when
                     Combine :: fun((X, Y) -> T),
                     List1 :: [X],
                     List2 :: [Y],
                     List3 :: [T],
                     X :: term(),
                     Y :: term(),
                     T :: term().

Equivalent to zipwith(Combine, List1, List2, fail).

zipwith(Combine, List1, List2, How)

(since OTP 26.0)

-spec zipwith(Combine, List1, List2, How) -> List3
                 when
                     Combine :: fun((X | DefaultX, Y | DefaultY) -> T),
                     List1 :: [X],
                     List2 :: [Y],
                     List3 :: [T],
                     X :: term(),
                     Y :: term(),
                     How :: fail | trim | {pad, {DefaultX, DefaultY}},
                     DefaultX :: term(),
                     DefaultY :: term(),
                     T :: term().

Combines the elements of two lists into a single list using the Combine fun.

For each pair X, Y of list elements from the two lists, the element in the result list is Combine(X, Y).

For a description of the How parameter, see zip/3.

zipwith(fun(X, Y) -> {X,Y} end, List1, List2) is equivalent to zip(List1, List2).

Examples

1> lists:zipwith(fun(X, Y) -> X+Y end, [1,2,3], [4,5,6], fail).
[5,7,9]

maps

Maps processing functions.

This module contains functions for maps processing. The Efficiency Guide contains a chapter that describes how to use maps efficiently.

Summary

Types

iterator()

iterator(Map, Order)

An iterator representing the associations in a map with keys of type Key and values of type Value.

iterator_order()

iterator_order(Key)

Key-based iterator order option that can be one of undefined (default for maps:iterator/1), ordered (sorted in map-key order), reversed (sorted in reverse map-key order), or a custom sorting function.

Functions

filter(Pred, MapOrIter)

Returns a map Map where each key-value pair from MapOrIter satisfies the predicate Pred(Key, Value).

filtermap(Fun, MapOrIter)

Calls Fun(Key, Value1) on each key-value pair of MapOrIter to update or remove associations from MapOrIter.

find(Key, Map)

Returns a tuple {ok, Value}, where Value is the value associated with Key, or error if no value is associated with Key in Map.

fold(Fun, Init, MapOrIter)

Calls Fun(Key, Value, AccIn) for every Key to value Value association in MapOrIter, starting with AccIn bound to Acc0.

foreach(Fun, MapOrIter)

Calls Fun(Key, Value) for every Key to Value association in MapOrIter.

from_keys(Keys, Value)

Takes a list of keys and a value and builds a map where all keys are associated with the same value.

from_list(List)

Takes a list of key-value tuples and builds a map.

get(Key, Map)

Returns value Value associated with Key if Map contains Key.

get(Key, Map, Default)

Returns the value associated with key Key in Map, or Default if Key is not present in the map.

groups_from_list(KeyFun, List)

Partitions the given List into a map of groups.

groups_from_list(KeyFun, ValueFun, List)

Partitions the given List into a map of groups.

intersect(Map1, Map2)

Computes the intersection of maps Map1 and Map2, producing a single map Map3.

intersect_with(Combiner, Map1, Map2)

Computes the intersection of maps Map1 and Map2, producing a single map Map3, where values having the same key are combined using the Combiner fun.

is_key(Key, Map)

Returns true if map Map contains Key; otherwise, returns false.

iterator(Map)

Returns a map iterator Iterator that can be used by maps:next/1 to traverse the key-value associations in a map.

iterator(Map, Order)

Returns a map iterator Iterator that can be used by maps:next/1 to traverse the key-value associations in a map sorted by key using the given Order.

keys(Map)

Returns a complete list of keys contained in Map, in any order.

map(Fun, MapOrIter)

Produces a new map Map by calling function Fun(Key, Value1) for every Key to value Value1 association in MapOrIter.

merge(Map1, Map2)

Merges maps Map1 and Map2 into a single map Map3, where values from Map2 override those from Map1 for duplicate keys.

merge_with(Combiner, Map1, Map2)

Merges maps Map1 and Map2 into a single map Map3, combining values for duplicate keys using the Combiner fun.

new()

Returns a new empty map.

next(Iterator)

Returns the next key-value association in Iterator and a new iterator for the remaining associations in the iterator.

put(Key, Value, Map1)

Associates Key with Value in Map1, replacing any existing value, and returns a new map Map2 with the updated association alongside the original entries from Map1.

remove(Key, Map1)

Removes Key and its associated value from Map1, if it exists, and returns a new map Map2 without Key.

size(Map)

Returns the number of key-value associations in Map.

take(Key, Map1)

Removes Key and its associated value from Map1, if it exists, returning a tuple with the removed value Value and the new map Map2; otherwise, returns error.

to_list(MapOrIterator)

Returns a list of pairs representing the key-value associations of MapOrIterator.

update(Key, Value, Map1)

If Key exists in Map1, its value is replaced with Value, and the function returns a new map Map2 with the updated association.

update_with(Key, Fun, Map1)

Updates a value in a Map1 associated with Key by calling Fun on the old value to produce a new value.

update_with(Key, Fun, Init, Map1)

Updates the value in Map1 for Key by applying Fun to the old value or using Init if Key is not present in the map.

values(Map)

Returns a complete list of values contained in map Map, in any order.

with(Ks, Map1)

Returns a new map Map2 with the keys K1 through Kn and their associated values from map Map1.

without(Ks, Map1)

Returns a new map Map2 without keys K1 through Kn and their associated values from map Map1.

Types

iterator()

(since OTP 17.0)

-type iterator() :: iterator(term(), term()).

iterator(Map, Order)

(since OTP 17.0)

-opaque iterator(Key, Value)

An iterator representing the associations in a map with keys of type Key and values of type Value.

Created using maps:iterator/1 or maps:iterator/2.

Consumed by:

maps:next/1

maps:filter/2

maps:filtermap/2

maps:fold/3

maps:foreach/2

maps:map/2

maps:to_list/1

iterator_order()

(since OTP 17.0)

-type iterator_order() :: iterator_order(term()).

iterator_order(Key)

(since OTP 17.0)

-type iterator_order(Key) :: undefined | ordered | reversed | fun((A :: Key, B :: Key) -> boolean()).

Key-based iterator order option that can be one of undefined (default for maps:iterator/1), ordered (sorted in map-key order), reversed (sorted in reverse map-key order), or a custom sorting function.

Used by maps:iterator/2.

The Expressions section contains descriptions of how terms are ordered.

Functions

filter(Pred, MapOrIter)

(since OTP 18.0)

-spec filter(Pred, MapOrIter) -> Map
                when
                    Pred :: fun((Key, Value) -> boolean()),
                    MapOrIter :: #{Key => Value} | iterator(Key, Value),
                    Map :: #{Key => Value}.

Returns a map Map where each key-value pair from MapOrIter satisfies the predicate Pred(Key, Value).

Unless MapOrIter is an ordered iterator returned by iterator/2, the order of the Pred(Key, Value) calls is not defined.

The call fails with a {badmap,Map} exception if MapOrIter is not a map or valid iterator, or with badarg if Pred is not a function of arity 2.

Examples

1> M = #{a => 2, b => 3, "a" => 1, "b" => 2}.
2> Pred = fun(K, V) -> is_atom(K) andalso V rem 2 =:= 0 end.
3> maps:filter(Pred, M).
#{a => 2}

filtermap(Fun, MapOrIter)

(since OTP 24.0)

-spec filtermap(Fun, MapOrIter) -> Map
                   when
                       Fun :: fun((Key, Value1) -> boolean() | {true, Value2}),
                       MapOrIter :: #{Key => Value1} | iterator(Key, Value1),
                       Map :: #{Key => Value1 | Value2}.

Calls Fun(Key, Value1) on each key-value pair of MapOrIter to update or remove associations from MapOrIter.

If Fun(Key, Value1) returns true, the association is copied to the result map. If it returns false, the association is not copied. If it returns {true, NewValue}, the value for Key is replaced with NewValue in the result map.

Unless MapOrIter is an ordered iterator returned by iterator/2, the order of the Fun(Key, Value1) calls is not defined.

The call fails with a {badmap,Map} exception if MapOrIter is not a map or valid iterator, or with badarg if Fun is not a function of arity 2.

Examples

1> Fun = fun(K, V) when is_atom(K) -> {true, V*2};
            (_, V) -> V rem 2 =:= 0
   end.
2> Map = #{k1 => 1, "k2" => 2, "k3" => 3}.
3> maps:filtermap(Fun, Map).
#{k1 => 2,"k2" => 2}

find(Key, Map)

(since OTP 17.0)

-spec find(Key, Map) -> {ok, Value} | error when Map :: #{Key => Value, _ => _}.

Returns a tuple {ok, Value}, where Value is the value associated with Key, or error if no value is associated with Key in Map.

The call fails with a {badmap,Map} exception if Map is not a map.

Examples

1> Map = #{"hi" => 42}.
2> Key = "hi".
3> maps:find(Key, Map).
{ok,42}

fold(Fun, Init, MapOrIter)

(since OTP 17.0)

-spec fold(Fun, Init, MapOrIter) -> Acc
              when
                  Fun :: fun((Key, Value, AccIn) -> AccOut),
                  Init :: term(),
                  Acc :: AccOut,
                  AccIn :: Init | AccOut,
                  MapOrIter :: #{Key => Value} | iterator(Key, Value).

Calls Fun(Key, Value, AccIn) for every Key to value Value association in MapOrIter, starting with AccIn bound to Acc0.

The Fun/3 fun must return a new accumulator, which is passed to the next call. The function returns the final value of the accumulator. The initial accumulator value Init is returned if the map is empty.

Unless MapOrIter is an ordered iterator returned by iterator/2, the order of the Fun(Key, Value, AccIn) calls is not defined.

The call fails with a {badmap,Map} exception if MapOrIter is not a map or valid iterator, or with badarg if Fun is not a function of arity 3.

Examples

1> Fun = fun(K, V, AccIn) -> AccIn + V end.
2> Map = #{k1 => 1, k2 => 2, k3 => 3}.
3> maps:fold(Fun, 0, Map).
6

foreach(Fun, MapOrIter)

(since OTP 24.0)

-spec foreach(Fun, MapOrIter) -> ok
                 when
                     Fun :: fun((Key, Value) -> term()),
                     MapOrIter :: #{Key => Value} | iterator(Key, Value).

Calls Fun(Key, Value) for every Key to Value association in MapOrIter.

Unless MapOrIter is an ordered iterator returned by iterator/2, the order of the Fun(Key, Value) calls is not defined.

The call fails with a {badmap,Map} exception if MapOrIter is not a map or valid iterator, or with badarg if Fun is not a function of arity 2.

Examples

1> Fun = fun(K, V) -> self() ! {K,V} end.
2> Map = #{p => 1, q => 2,x => 10, y => 20, z => 30}.
3> maps:foreach(Fun, maps:iterator(Map, ordered)).
ok
4> [receive X -> X end || _ <- [1,2,3,4,5]].
[{p,1},{q,2},{x,10},{y,20},{z,30}]

from_keys(Keys, Value)

(since OTP 24.0)

-spec from_keys(Keys, Value) -> Map when Keys :: list(), Value :: term(), Map :: map().

Takes a list of keys and a value and builds a map where all keys are associated with the same value.

Examples

1> Keys = ["a", "b", "c"].
2> maps:from_keys(Keys, ok).
#{"a" => ok,"b" => ok,"c" => ok}

from_list(List)

(since OTP 17.0)

-spec from_list(List) -> Map when List :: [{Key, Value}], Key :: term(), Value :: term(), Map :: map().

Takes a list of key-value tuples and builds a map.

If the same key appears more than once, the last (rightmost) value is used, and previous values are ignored.

Examples

1> List = [{"a",ignored},{1337,"value two"},{42,value_three},{"a",1}].
2> maps:from_list(List).
#{42 => value_three,1337 => "value two","a" => 1}

get(Key, Map)

(since OTP 17.0)

-spec get(Key, Map) -> Value when Key :: term(), Map :: map(), Value :: term().

Returns value Value associated with Key if Map contains Key.

The call fails with a {badmap,Map} exception if Map is not a map, or with a {badkey,Key} exception if no value is associated with Key.

Examples

1> Key = 1337.
2> Map = #{42 => value_two,1337 => "value one","a" => 1}.
3> maps:get(Key, Map).
"value one"

get(Key, Map, Default)

(since OTP 17.1)

-spec get(Key, Map, Default) -> Value | Default when Map :: #{Key => Value, _ => _}.

Returns the value associated with key Key in Map, or Default if Key is not present in the map.

The call fails with a {badmap,Map} exception if Map is not a map.

Examples

1> Map = #{key1 => val1, key2 => val2}.
#{key1 => val1,key2 => val2}
2> maps:get(key1, Map, "Default value").
val1
3> maps:get(key3, Map, "Default value").
"Default value"

groups_from_list(KeyFun, List)

(since OTP 25.0)

-spec groups_from_list(KeyFun, List) -> GroupsMap
                          when
                              KeyFun :: fun((Elem) -> Key),
                              GroupsMap :: #{Key => Group},
                              Key :: term(),
                              List :: [Elem],
                              Group :: [Elem],
                              Elem :: term().

Partitions the given List into a map of groups.

The result is a map where each key is given by KeyFun and each value is a list of elements from the given List for which KeyFun returned the same key.

The order of elements within each group list is preserved from the original list.

Examples

1> EvenOdd = fun(X) when X rem 2 =:= 0 -> even;
                (_) -> odd
             end.
2> maps:groups_from_list(EvenOdd, [1, 2, 3]).
#{even => [2], odd => [1, 3]}
3> maps:groups_from_list(fun length/1, ["ant", "buffalo", "cat", "dingo"]).
#{3 => ["ant", "cat"], 5 => ["dingo"], 7 => ["buffalo"]}

groups_from_list(KeyFun, ValueFun, List)

(since OTP 25.0)

-spec groups_from_list(KeyFun, ValueFun, List) -> GroupsMap
                          when
                              KeyFun :: fun((Elem) -> Key),
                              ValueFun :: fun((Elem) -> Value),
                              GroupsMap :: #{Key := Group},
                              Key :: term(),
                              Value :: term(),
                              List :: [Elem],
                              Group :: [Value],
                              Elem :: term().

Partitions the given List into a map of groups.

The result is a map where each key is given by KeyFun and each value is a list of elements from the given List, mapped via ValueFun, for which KeyFun returned the same key.

The order of elements within each group list is preserved from the original list.

Examples

1> EvenOdd = fun(X) -> case X rem 2 of 0 -> even; 1 -> odd end end.
2> Square = fun(X) -> X * X end.
3> maps:groups_from_list(EvenOdd, Square, [1, 2, 3]).
#{even => [4], odd => [1, 9]}
4> maps:groups_from_list(
    fun length/1,
    fun lists:reverse/1,
    ["ant", "buffalo", "cat", "dingo"]).
#{3 => ["tna", "tac"],5 => ["ognid"],7 => ["olaffub"]}

intersect(Map1, Map2)

(since OTP 24.0)

-spec intersect(Map1, Map2) -> Map3
                   when Map1 :: #{Key => term()}, Map2 :: #{term() => Value2}, Map3 :: #{Key => Value2}.

Computes the intersection of maps Map1 and Map2, producing a single map Map3.

If a key exists in both maps, the value in Map1 is superseded by the value in Map2. Keys existing in only one of the maps are discarded along with their values.

The call fails with a {badmap,Map} exception if Map1 or Map2 is not a map.

Examples

1> Map1 = #{a => "one", b => "two"}.
2> Map2 = #{a => 1, c => 3}.
3> maps:intersect(Map1, Map2).
#{a => 1}

intersect_with(Combiner, Map1, Map2)

(since OTP 24.0)

-spec intersect_with(Combiner, Map1, Map2) -> Map3
                        when
                            Map1 :: #{Key => Value1},
                            Map2 :: #{term() => Value2},
                            Combiner :: fun((Key, Value1, Value2) -> CombineResult),
                            Map3 :: #{Key => CombineResult}.

Computes the intersection of maps Map1 and Map2, producing a single map Map3, where values having the same key are combined using the Combiner fun.

When Combiner is applied, the key that exists in both maps is the first parameter, the value from Map1 is the second parameter, and the value from Map2 is the third parameter.

The call fails with a {badmap,Map} exception if Map1 or Map2 is not a map. The call fails with a badarg exception if Combiner is not a fun that takes three arguments.

Examples

1> Map1 = #{a => "one", b => "two"}.
2> Map2 = #{a => 1, c => 3}.
3> maps:intersect_with(fun(_Key, Val1, Val2) -> {Val1, Val2} end, Map1, Map2).
#{a => {"one",1}}

is_key(Key, Map)

(since OTP 17.0)

-spec is_key(Key, Map) -> boolean() when Key :: term(), Map :: map().

Returns true if map Map contains Key; otherwise, returns false.

The call fails with a {badmap,Map} exception if Map is not a map.

Examples

1> Map = #{"42" => value}.
#{"42" => value}
2> maps:is_key("42", Map).
true
3> maps:is_key(value, Map).
false

iterator(Map)

(since OTP 21.0)

-spec iterator(Map) -> Iterator when Map :: #{Key => Value}, Iterator :: iterator(Key, Value).

Returns a map iterator Iterator that can be used by maps:next/1 to traverse the key-value associations in a map.

The order of iteration is undefined. When iterating over a map, the memory usage is guaranteed to be bounded no matter the size of the map.

The call fails with a {badmap,Map} exception if Map is not a map.

Examples

1> M = #{ "foo" => 1, "bar" => 2 }.
#{"foo" => 1,"bar" => 2}
2> I = maps:iterator(M).
3> {K1, V1, I2} = maps:next(I), {K1, V1}.
{"bar",2}
4> {K2, V2, I3} = maps:next(I2),{K2, V2}.
{"foo",1}
5> maps:next(I3).
none

iterator(Map, Order)

(since OTP 26.0)

-spec iterator(Map, Order) -> Iterator
                  when
                      Map :: #{Key => Value},
                      Order :: iterator_order(Key),
                      Iterator :: iterator(Key, Value).

Returns a map iterator Iterator that can be used by maps:next/1 to traverse the key-value associations in a map sorted by key using the given Order.

The call fails with a {badmap,Map} exception if Map is not a map, or with a badarg exception if Order is invalid.

Examples

Ordered iterator:

1> M = #{a => 1, b => 2}.
2> OrdI = maps:iterator(M, ordered).
3> {K1, V1, OrdI2} = maps:next(OrdI), {K1, V1}.
{a,1}
4> {K2, V2, OrdI3} = maps:next(OrdI2),{K2, V2}.
{b,2}
5> maps:next(OrdI3).
none

Iterator ordered in reverse:

1> M = #{a => 1, b => 2}.
2> RevI = maps:iterator(M, reversed).
3> {K2, V2, RevI2} = maps:next(RevI), {K2, V2}.
{b,2}
4> {K1, V1, RevI3} = maps:next(RevI2),{K1, V1}.
{a,1}
5> maps:next(RevI3).
none
6> maps:to_list(RevI).
[{b,2},{a,1}]

Using a custom ordering function that orders binaries by size:

1> M = #{<<"abcde">> => d, <<"y">> => b, <<"x">> => a, <<"pqr">> => c}.
2> SizeI = fun(A, B) when byte_size(A) < byte_size(B) -> true;
              (A, B) when byte_size(A) > byte_size(B) -> false;
              (A, B) -> A =< B
           end.
3> SizeOrdI = maps:iterator(M, SizeI).
4> maps:to_list(SizeOrdI).
[{<<"x">>,a},{<<"y">>,b},{<<"pqr">>,c},{<<"abcde">>,d}]

keys(Map)

(since OTP 17.0)

-spec keys(Map) -> Keys when Map :: #{Key => _}, Keys :: [Key].

Returns a complete list of keys contained in Map, in any order.

The call fails with a {badmap,Map} exception if Map is not a map.

Examples

1> Map = #{42 => three,1337 => "two","a" => 1}.
2> maps:keys(Map).
[42,1337,"a"]

map(Fun, MapOrIter)

(since OTP 17.0)

-spec map(Fun, MapOrIter) -> Map
             when
                 Fun :: fun((Key, Value1) -> Value2),
                 MapOrIter :: #{Key => Value1} | iterator(Key, Value1),
                 Map :: #{Key => Value2}.

Produces a new map Map by calling function Fun(Key, Value1) for every Key to value Value1 association in MapOrIter.

The Fun/2 fun must return value Value2 to be associated with key Key for the new map Map.

Unless MapOrIter is an ordered iterator returned by iterator/2, the order of the Fun(Key, Value1) calls is not defined.

The call fails with a {badmap,Map} exception if MapOrIter is not a map or valid iterator, or with badarg if Fun is not a function of arity 2.

Examples

1> Fun = fun(K,V1) when is_list(K) -> V1*2 end.
2> Map = #{"k1" => 1, "k2" => 2, "k3" => 3}.
3> maps:map(Fun, Map).
#{"k1" => 2,"k2" => 4,"k3" => 6}

merge(Map1, Map2)

(since OTP 17.0)

-spec merge(Map1, Map2) -> Map3 when Map1 :: map(), Map2 :: map(), Map3 :: map().

Merges maps Map1 and Map2 into a single map Map3, where values from Map2 override those from Map1 for duplicate keys.

The call fails with a {badmap,Map} exception if Map1 or Map2 is not a map.

Examples

1> Map1 = #{a => "one", b => "two"}.
2> Map2 = #{a => 1, c => 3}.
3> maps:merge(Map1, Map2).
#{a => 1,b => "two",c => 3}

merge_with(Combiner, Map1, Map2)

(since OTP 24.0)

-spec merge_with(Combiner, Map1, Map2) -> Map3
                    when
                        Map1 :: #{Key1 => Value1},
                        Map2 :: #{Key2 => Value2},
                        Combiner :: fun((Key1, Value1, Value2) -> CombineResult),
                        Map3 :: #{Key1 => CombineResult, Key1 => Value1, Key2 => Value2}.

Merges maps Map1 and Map2 into a single map Map3, combining values for duplicate keys using the Combiner fun.

When Combiner is applied, the key that exists in both maps is the first parameter, the value from Map1 is the second parameter, and the value from Map2 is the third parameter.

The call fails with a {badmap,Map} exception if Map1 or Map2 is not a map. The call fails with a badarg exception if Combiner is not a fun that takes three arguments.

Examples

1> Map1 = #{a => 3, b => 5}.
2> Map2 = #{a => 4, c => 17}.
3> maps:merge_with(fun(_Key, Val1, Val2) -> Val1 + Val2 end, Map1, Map2).
#{a => 7,b => 5,c => 17}

new()

(since OTP 17.0)

-spec new() -> Map when Map :: #{}.

Returns a new empty map.

Examples

1> maps:new().
#{}

next(Iterator)

(since OTP 21.0)

-spec next(Iterator) -> {Key, Value, NextIterator} | none
              when Iterator :: iterator(Key, Value), NextIterator :: iterator(Key, Value).

Returns the next key-value association in Iterator and a new iterator for the remaining associations in the iterator.

If there are no more associations in the iterator, none is returned.

Examples

1> Map = #{a => 1, b => 2, c => 3}.
#{a => 1,b => 2,c => 3}
2> I = maps:iterator(Map, ordered).
3> {K1, V1, I1} = maps:next(I), {K1, V1}.
{a,1}
4> {K2, V2, I2} = maps:next(I1), {K2, V2}.
{b,2}
5> {K3, V3, I3} = maps:next(I2), {K3, V3}.
{c,3}
6> maps:next(I3).
none

put(Key, Value, Map1)

(since OTP 17.0)

-spec put(Key, Value, Map1) -> Map2 when Key :: term(), Value :: term(), Map1 :: map(), Map2 :: map().

Associates Key with Value in Map1, replacing any existing value, and returns a new map Map2 with the updated association alongside the original entries from Map1.

The call fails with a {badmap,Map} exception if Map1 is not a map.

Examples

1> Map = #{"a" => 1}.
#{"a" => 1}
2> maps:put("a", 42, Map).
#{"a" => 42}
3> maps:put("b", 1337, Map).
#{"a" => 1,"b" => 1337}

remove(Key, Map1)

(since OTP 17.0)

-spec remove(Key, Map1) -> Map2 when Key :: term(), Map1 :: map(), Map2 :: map().

Removes Key and its associated value from Map1, if it exists, and returns a new map Map2 without Key.

The call fails with a {badmap,Map} exception if Map1 is not a map.

Examples

1> Map = #{"a" => 1}.
#{"a" => 1}
2> maps:remove("a", Map).
#{}
3> maps:remove("b", Map).
#{"a" => 1}

size(Map)

(since OTP 17.0)

-spec size(Map) -> non_neg_integer() when Map :: map().

Returns the number of key-value associations in Map.

This operation occurs in constant time.

Examples

1> Map = #{42 => value_two,1337 => "value one","a" => 1}.
2> maps:size(Map).
3

take(Key, Map1)

(since OTP 19.0)

-spec take(Key, Map1) -> {Value, Map2} | error when Map1 :: #{Key => Value, _ => _}, Map2 :: #{_ => _}.

Removes Key and its associated value from Map1, if it exists, returning a tuple with the removed value Value and the new map Map2; otherwise, returns error.

The call will fail with a {badmap,Map} exception if Map1 is not a map.

Example:

1> Map = #{"a" => "hello", "b" => "world"}.
#{"a" => "hello", "b" => "world"}
2> maps:take("a", Map).
{"hello",#{"b" => "world"}}
3> maps:take("does not exist", Map).
error

to_list(MapOrIterator)

(since OTP 17.0)

-spec to_list(MapOrIterator) -> [{Key, Value}]
                 when MapOrIterator :: #{Key => Value} | iterator(Key, Value).

Returns a list of pairs representing the key-value associations of MapOrIterator.

Unless MapOrIter is an ordered iterator returned by iterator/2, the order of the {Key, Value} tuples in the resulting list is not defined.

The call fails with a {badmap,Map} exception if MapOrIterator is not a map or an iterator obtained by a call to iterator/1 or iterator/2.

Examples

1> Map = #{42 => value_three,1337 => "value two","a" => 1}.
2> maps:to_list(Map).
[{42,value_three},{1337,"value two"},{"a",1}]

Using an ordered iterator to return an ordered list:

1> Map = #{z => 1, y => 2, x => 3}.
2> maps:to_list(maps:iterator(Map, ordered)).
[{x,3},{y,2},{z,1}]

update(Key, Value, Map1)

(since OTP 17.0)

-spec update(Key, Value, Map1) -> Map2 when Map1 :: #{Key := _, _ => _}, Map2 :: #{Key := Value, _ => _}.

If Key exists in Map1, its value is replaced with Value, and the function returns a new map Map2 with the updated association.

The call fails with a {badmap,Map} exception if Map1 is not a map, or with a {badkey,Key} exception if no value is associated with Key.

Examples

1> Map = #{"a" => 1}.
#{"a" => 1}
2> maps:update("a", 42, Map).
#{"a" => 42}

update_with(Key, Fun, Map1)

(since OTP 19.0)

-spec update_with(Key, Fun, Map1) -> Map2
                     when
                         Map1 :: #{Key := Value1, _ => _},
                         Map2 :: #{Key := Value2, _ => _},
                         Fun :: fun((Value1) -> Value2).

Updates a value in a Map1 associated with Key by calling Fun on the old value to produce a new value.

The call fails with a {badkey,Key} exception if Key is not present in the map.

Examples

1> Map = #{counter => 1}.
2> Fun = fun(V) -> V + 1 end.
3> maps:update_with(counter, Fun, Map).
#{counter => 2}

update_with(Key, Fun, Init, Map1)

(since OTP 19.0)

-spec update_with(Key, Fun, Init, Map1) -> Map2
                     when
                         Map1 :: #{Key => Value1, _ => _},
                         Map2 :: #{Key := Value2 | Init, _ => _},
                         Fun :: fun((Value1) -> Value2).

Updates the value in Map1 for Key by applying Fun to the old value or using Init if Key is not present in the map.

Examples

1> Map = #{"counter" => 1}.
2> Fun = fun(V) -> V + 1 end.
3> maps:update_with("counter", Fun, 42, Map).
#{"counter" => 2}
4> maps:update_with("new counter", Fun, 42, Map).
#{"counter" => 1,"new counter" => 42}

values(Map)

(since OTP 17.0)

-spec values(Map) -> Values when Map :: #{_ => Value}, Values :: [Value].

Returns a complete list of values contained in map Map, in any order.

The call fails with a {badmap,Map} exception if Map is not a map.

Examples

1> Map = #{42 => value_three,1337 => "value two","a" => 1}.
2> maps:values(Map).
[value_three,"value two",1]

with(Ks, Map1)

(since OTP 17.3)

-spec with(Ks, Map1) -> Map2 when Ks :: [K], Map1 :: #{K => V, _ => _}, Map2 :: #{K => V}.

Returns a new map Map2 with the keys K1 through Kn and their associated values from map Map1.

Any key in Ks that does not exist in Map1 is ignored.

Examples

1> Map = #{42 => value_three,1337 => "value two","a" => 1}.
2> Keys = ["a",42,"other key"].
3> maps:with(Keys, Map).
#{42 => value_three,"a" => 1}

without(Ks, Map1)

(since OTP 17.0)

-spec without(Ks, Map1) -> Map2 when Ks :: [K], Map1 :: map(), Map2 :: map(), K :: term().

Returns a new map Map2 without keys K1 through Kn and their associated values from map Map1.

Any key in Ks that does not exist in Map1 is ignored.

Examples

1> Map = #{42 => value_three, 1337 => "value two", "a" => 1}.
2> Keys = ["a",42,"other key"].
3> maps:without(Keys, Map).
#{1337 => "value two"}

math

Mathematical functions.

This module provides an interface to a number of mathematical functions.

Summary

Functions

acos(X)

Returns the arc cosine of X in radians.

acosh(X)

Returns the inverse hyperbolic cosine of X.

asin(X)

Returns the arc cosine of X in radians.

asinh(X)

Returns the inverse hyperbolic sine of X.

atan2(Y, X)

Returns the arc tangent of Y/X in radians, using the signs of both arguments to determine the quadrant of the return value.

atan(X)

Returns the arc tangent of X in radians.

atanh(X)

Returns the inverse hyperbolic tangent of X.

ceil(X)

Returns the ceiling of X.

cos(X)

Returns the cosine of X in radians.

cosh(X)

Returns the hyperbolic cosine of X.

erf(X)

Returns the error function of X.

erfc(X)

Returns 1.0 - erf(X), computed using methods that avoid cancellation for large X.

exp(X)

Returns e raised to the power of X.

floor(X)

Returns the floor of X.

fmod(X, Y)

Returns the floating point remainder X divided by Y.

log2(X)

Returns logarithm of X to base 2.

log10(X)

Returns logarithm of X to base 10.

log(X)

Returns the natural logarithm of X.

pi()

Returns the ratio of the circumference of a circle to its diameter.

pow(X, N)

Raise X to the power N.

sin(X)

Returns the sine of X in radians.

sinh(X)

Returns the hyperbolic sine of X.

sqrt(X)

Returns the non-negative square root of X.

tan(X)

Returns the tangent of X in radians.

tanh(X)

Returns the hyperbolic tangent of X.

tau()

Returns the ratio of the circumference of a circle to its radius.

Functions

acos(X)

-spec acos(X) -> float() when X :: number().

Returns the arc cosine of X in radians.

Examples

1> math:acos(1.0).
0.0

acosh(X)

-spec acosh(X) -> float() when X :: number().

Returns the inverse hyperbolic cosine of X.

Examples

1> math:acosh(1.0).
0.0

asin(X)

-spec asin(X) -> float() when X :: number().

Returns the arc cosine of X in radians.

Examples

1> math:asin(0.0).
0.0

asinh(X)

-spec asinh(X) -> float() when X :: number().

Returns the inverse hyperbolic sine of X.

Examples

1> math:asinh(0.0).
0.0

atan2(Y, X)

-spec atan2(Y, X) -> float() when Y :: number(), X :: number().

Returns the arc tangent of Y/X in radians, using the signs of both arguments to determine the quadrant of the return value.

Examples

1> math:atan2(0.0, -10.0).
3.141592653589793

atan(X)

-spec atan(X) -> float() when X :: number().

Returns the arc tangent of X in radians.

Examples

1> math:atan(0.0).
0.0

atanh(X)

-spec atanh(X) -> float() when X :: number().

Returns the inverse hyperbolic tangent of X.

Examples

1> math:atanh(0.0).
0.0

ceil(X)

(since OTP 20.0)

-spec ceil(X) -> float() when X :: number().

Returns the ceiling of X.

Examples

1> math:ceil(7.5).
8.0
2> math:ceil(-5.5).
-5.0
3> math:ceil(1.0).
1.0

cos(X)

-spec cos(X) -> float() when X :: number().

Returns the cosine of X in radians.

Examples

1> math:cos(0.0)
1.0

cosh(X)

-spec cosh(X) -> float() when X :: number().

Returns the hyperbolic cosine of X.

Examples

1> math:cosh(0.0)
1.0

erf(X)

-spec erf(X) -> float() when X :: number().

Returns the error function of X.

See Error function (Wikipedia).

Examples

1> math:erf(0.0).
0.0
2> math:erf(10.0).
1.0

erfc(X)

-spec erfc(X) -> float() when X :: number().

Returns 1.0 - erf(X), computed using methods that avoid cancellation for large X.

Examples

1> math:erfc(0.0).
1.0

exp(X)

-spec exp(X) -> float() when X :: number().

Returns e raised to the power of X.

Examples

1> math:exp(0).
1.0
2> trunc(100 * math:exp(1)).
271

floor(X)

(since OTP 20.0)

-spec floor(X) -> float() when X :: number().

Returns the floor of X.

Examples

1> math:floor(9.1).
9.0
2> math:floor(-1.5).
-2.0
3> math:floor(1.0)
1.0

fmod(X, Y)

(since OTP 20.0)

-spec fmod(X, Y) -> float() when X :: number(), Y :: number().

Returns the floating point remainder X divided by Y.

Examples

1> math:fmod(10.5, 8.0).
2.5

log2(X)

(since OTP 18.0)

-spec log2(X) -> float() when X :: number().

Returns logarithm of X to base 2.

Examples

1> math:log2(1.0).
0.0
2> math:log2(2.0).
1.0
3> math:log2(64).
6.0

log10(X)

-spec log10(X) -> float() when X :: number().

Returns logarithm of X to base 10.

Examples

1> math:log10(1.0).
0.0
2> math:log10(10.0).
1.0
3> math:log10(100).
2.0

log(X)

-spec log(X) -> float() when X :: number().

Returns the natural logarithm of X.

Examples

1> math:log(1.0).
0.0
2> math:log(2.718281828459045).
1.0

pi()

-spec pi() -> float().

Returns the ratio of the circumference of a circle to its diameter.

Examples

1> math:pi().
3.141592653589793

pow(X, N)

-spec pow(X, N) -> float() when X :: number(), N :: number().

Raise X to the power N.

Examples

1> math:pow(2, 6).
64.0
2> math:pow(10.0, 3.0).
1000.0

sin(X)

-spec sin(X) -> float() when X :: number().

Returns the sine of X in radians.

Examples

1> math:sin(0.0)
0.0

sinh(X)

-spec sinh(X) -> float() when X :: number().

Returns the hyperbolic sine of X.

Examples

1> math:sinh(0.0)
0.0

sqrt(X)

-spec sqrt(X) -> float() when X :: number().

Returns the non-negative square root of X.

Examples

1> math:sqrt(2).
1.4142135623730951
2> math:sqrt(100.0).
10.0

tan(X)

-spec tan(X) -> float() when X :: number().

Returns the tangent of X in radians.

Examples

1> math:tan(0.0)
0.0

tanh(X)

-spec tanh(X) -> float() when X :: number().

Returns the hyperbolic tangent of X.

Examples

1> math:tan(0.0)
0.0

tau()

(since OTP 26.0)

-spec tau() -> float().

Returns the ratio of the circumference of a circle to its radius.

This constant is equivalent to a full turn when described in radians.

Examples

1> math:tau().
6.283185307179586
2> math:tau() == 2 * math:pi().
true

calendar

Local and universal time, day of the week, date and time conversions.

This module provides computation of local and universal time, day of the week, and many time conversion functions.

Time is local when it is adjusted in accordance with the current time zone and daylight saving. Time is universal when it reflects the time at longitude zero, without any adjustment for daylight saving. Universal Coordinated Time (UTC) time is also called Greenwich Mean Time (GMT).

The time functions local_time/0 and universal_time/0 in this module both return date and time. This is because separate functions for date and time can result in a date/time combination that is displaced by 24 hours. This occurs if one of the functions is called before midnight, and the other after midnight. This problem also applies to the Erlang BIFs date/0 and time/0, and their use is strongly discouraged if a reliable date/time stamp is required.

All dates conform to the Gregorian calendar. This calendar was introduced by Pope Gregory XIII in 1582 and was used in all Catholic countries from this year. Protestant parts of Germany and the Netherlands adopted it in 1698, England followed in 1752, and Russia in 1918 (the October revolution of 1917 took place in November according to the Gregorian calendar).

The Gregorian calendar in this module is extended back to year 0. For a given date, the gregorian days is the number of days up to and including the date specified. Similarly, the gregorian seconds for a specified date and time is the number of seconds up to and including the specified date and time.

For computing differences between epochs in time, use the functions counting gregorian days or seconds. If epochs are specified as local time, they must be converted to universal time to get the correct value of the elapsed time between epochs. Use of function time_difference/2 is discouraged.

Different definitions exist for the week of the year. This module contains a week of the year implementation conforming to the ISO 8601 standard. As the week number for a specified date can fall on the previous, the current, or on the next year, it is important to specify both the year and the week number. Functions iso_week_number/0 and iso_week_number/1 return a tuple of the year and the week number.

Leap Years

The notion that every fourth year is a leap year is not completely true. By the Gregorian rule, a year Y is a leap year if one of the following rules is valid:

Y is divisible by 4, but not by 100.

Y is divisible by 400.

Hence, 1996 is a leap year, 1900 is not, but 2000 is.

Date and Time Source

Local time is obtained from the Erlang BIF localtime/0. Universal time is computed from the BIF universaltime/0.

The following apply:

There are 86400 seconds in a day.

There are 365 days in an ordinary year.

There are 366 days in a leap year.

There are 1461 days in a 4 year period.

There are 36524 days in a 100 year period.

There are 146097 days in a 400 year period.

There are 719528 days between Jan 1, 0 and Jan 1, 1970.

Summary

Types

date()

A date using the Gregorian calendar.

datetime1970()

datetime()

day()

daynum()

hour()

ldom()

The last day of the month.

minute()

month()

offset()

rfc3339_string()

rfc3339_time_unit()

The time unit used by the rfc3339 conversion functions.

second()

secs_per_day()

time()

weeknum()

year1970()

year()

The year using the Gregorian calendar.

yearweeknum()

Functions

date_to_gregorian_days(Date)

Computes the number of gregorian days starting with year 0 and ending at the specified date.

date_to_gregorian_days(Year, Month, Day)

Equivalent to date_to_gregorian_days({Year, Month, Day}).

datetime_to_gregorian_seconds(DateTime)

Computes the number of gregorian seconds starting with year 0 and ending at the specified date and time.

day_of_the_week(Date)

Computes the day of the week from the specified Year, Month, and Day. Returns the day of the week as 1: Monday, 2: Tuesday, and so on.

day_of_the_week(Year, Month, Day)

Equivalent to day_of_the_week({Year, Month, Day}).

gregorian_days_to_date(Days)

Computes the date from the specified number of gregorian days.

gregorian_seconds_to_datetime(Seconds)

Computes the date and time from the specified number of gregorian seconds.

is_leap_year(Year)

Checks if the specified year is a leap year.

iso_week_number()

Returns tuple {Year, WeekNum} representing the ISO week number for the actual date. To determine the actual date, use function local_time/0.

iso_week_number(Date)

Returns tuple {Year, WeekNum} representing the ISO week number for the specified date.

last_day_of_the_month(Year, Month)

Computes the number of days in a month.

local_time()

Returns the local time reported by the underlying operating system.

local_time_to_system_time(LocalTime)

Equivalent to local_time_to_system_time(LocalTime, []).

local_time_to_system_time(LocalTime, Options)

Converts local time into system time. Error will occur if the local time is non existing or ambiguous due to DST, see calendar:local_time_to_universal_time_dst/1.

local_time_to_universal_time(DateTime1) deprecated

Converts from local time to Universal Coordinated Time (UTC). DateTime1 must refer to a local date after Jan 1, 1970.

local_time_to_universal_time_dst(DateTime1)

Converts from local time to Universal Coordinated Time (UTC). DateTime1 must refer to a local date after Jan 1, 1970.

now_to_datetime(Now)

Returns Universal Coordinated Time (UTC) converted from the return value from erlang:timestamp/0.

now_to_local_time(Now)

Returns local date and time converted from the return value from erlang:timestamp/0.

now_to_universal_time(Now)

Returns Universal Coordinated Time (UTC) converted from the return value from erlang:timestamp/0.

rfc3339_to_system_time(DateTimeString)

Equivalent to rfc3339_to_system_time(DateTimeString, []).

rfc3339_to_system_time(DateTimeString, Options)

Converts an RFC 3339 timestamp into system time. The data format of RFC 3339 timestamps is described by RFC 3339. Starting from OTP 25.1, the minutes part of the time zone is optional.

seconds_to_daystime(Seconds)

Converts a specified number of seconds into days, hours, minutes, and seconds. Time is always non-negative, but Days is negative if argument Seconds is.

seconds_to_time(Seconds)

Computes the time from the specified number of seconds. Seconds must be less than the number of seconds per day (86400).

system_time_to_local_time(Time, TimeUnit)

Converts a specified system time into local date and time.

system_time_to_rfc3339(Time)

Equivalent to system_time_to_rfc3339(Time, []).

system_time_to_rfc3339(Time, Options)

Converts a system time into an RFC 3339 timestamp.

system_time_to_universal_time(Time, TimeUnit)

Converts a specified system time into universal date and time.

time_difference(T1, T2)

Returns the difference between two {Date, Time} tuples. T2 is to refer to an epoch later than T1.

time_to_seconds(Time)

Returns the number of seconds since midnight up to the specified time.

universal_time()

Returns the Universal Coordinated Time (UTC) reported by the underlying operating system. Returns local time if universal time is unavailable.

universal_time_to_local_time(DateTime)

Converts from Universal Coordinated Time (UTC) to local time. DateTime must refer to a date after Jan 1, 1970.

universal_time_to_system_time(UniversalTime)

Equivalent to universal_time_to_system_time(LocalTime, []).

universal_time_to_system_time(DateTime, Options)

Converts universal time into system time.

valid_date(Date)

This function checks if a date is a valid.

valid_date(Year, Month, Day)

Equivalent to valid_date({Year, Month, Day}).

Types

date()

-type date() :: {year(), month(), day()}.

A date using the Gregorian calendar.

All APIs expect this to be a valid date. If the source of the date is unknown, then verify that is it valid by calling valid_date/1 before using it.

datetime1970()

-type datetime1970() :: {{year1970(), month(), day()}, time()}.

datetime()

-type datetime() :: {date(), time()}.

day()

(not exported)

-type day() :: 1..31.

daynum()

(not exported)

-type daynum() :: 1..7.

hour()

(not exported)

-type hour() :: 0..23.

ldom()

(not exported)

-type ldom() :: 28 | 29 | 30 | 31.

The last day of the month.

minute()

(not exported)

-type minute() :: 0..59.

month()

(not exported)

-type month() :: 1..12.

offset()

(not exported)

-type offset() :: [byte()] | (Time :: integer()).

rfc3339_string()

(not exported)

-type rfc3339_string() :: [byte(), ...] | binary().

rfc3339_time_unit()

(not exported)

-type rfc3339_time_unit() :: microsecond | millisecond | nanosecond | second | native.

The time unit used by the rfc3339 conversion functions.

Note

The native time unit was added to rfc3339_time_unit/0 in OTP 25.0.

second()

(not exported)

-type second() :: 0..59.

secs_per_day()

(not exported)

-type secs_per_day() :: 0..86399.

time()

-type time() :: {hour(), minute(), second()}.

weeknum()

(not exported)

-type weeknum() :: 1..53.

year1970()

(not exported)

-type year1970() :: 1970..10000.

year()

(not exported)

-type year() :: non_neg_integer().

The year using the Gregorian calendar.

Year cannot be abbreviated. For example, 93 denotes year 93, not 1993. The valid range depends on the underlying operating system.

yearweeknum()

(not exported)

-type yearweeknum() :: {year(), weeknum()}.

Functions

date_to_gregorian_days(Date)

-spec date_to_gregorian_days(Date) -> Days when Date :: date(), Days :: non_neg_integer().

Computes the number of gregorian days starting with year 0 and ending at the specified date.

date_to_gregorian_days(Year, Month, Day)

-spec date_to_gregorian_days(Year, Month, Day) -> Days
                                when
                                    Year :: year(),
                                    Month :: month(),
                                    Day :: day(),
                                    Days :: non_neg_integer().

Equivalent to date_to_gregorian_days({Year, Month, Day}).

datetime_to_gregorian_seconds(DateTime)

-spec datetime_to_gregorian_seconds(DateTime) -> Seconds
                                       when DateTime :: datetime(), Seconds :: non_neg_integer().

Computes the number of gregorian seconds starting with year 0 and ending at the specified date and time.

day_of_the_week(Date)

-spec day_of_the_week(Date) -> daynum() when Date :: date().

Computes the day of the week from the specified Year, Month, and Day. Returns the day of the week as 1: Monday, 2: Tuesday, and so on.

day_of_the_week(Year, Month, Day)

-spec day_of_the_week(Year, Month, Day) -> daynum() when Year :: year(), Month :: month(), Day :: day().

Equivalent to day_of_the_week({Year, Month, Day}).

gregorian_days_to_date(Days)

-spec gregorian_days_to_date(Days) -> date() when Days :: non_neg_integer().

Computes the date from the specified number of gregorian days.

gregorian_seconds_to_datetime(Seconds)

-spec gregorian_seconds_to_datetime(Seconds) -> datetime() when Seconds :: non_neg_integer().

Computes the date and time from the specified number of gregorian seconds.

is_leap_year(Year)

-spec is_leap_year(Year) -> boolean() when Year :: year().

Checks if the specified year is a leap year.

iso_week_number()

(since OTP R14B02)

-spec iso_week_number() -> yearweeknum().

Returns tuple {Year, WeekNum} representing the ISO week number for the actual date. To determine the actual date, use function local_time/0.

iso_week_number(Date)

(since OTP R14B02)

-spec iso_week_number(Date) -> yearweeknum() when Date :: date().

Returns tuple {Year, WeekNum} representing the ISO week number for the specified date.

last_day_of_the_month(Year, Month)

-spec last_day_of_the_month(Year, Month) -> LastDay
                               when Year :: year(), Month :: month(), LastDay :: ldom().

Computes the number of days in a month.

local_time()

-spec local_time() -> datetime().

Returns the local time reported by the underlying operating system.

local_time_to_system_time(LocalTime)

(since OTP 28.0)

-spec local_time_to_system_time(datetime1970()) -> pos_integer().

Equivalent to local_time_to_system_time(LocalTime, []).

local_time_to_system_time(LocalTime, Options)

(since OTP 28.0)

-spec local_time_to_system_time(datetime1970(), Options) -> pos_integer()
                                   when Options :: [Option], Option :: {unit, erlang:time_unit()}.

Converts local time into system time. Error will occur if the local time is non existing or ambiguous due to DST, see calendar:local_time_to_universal_time_dst/1.

local_time_to_universal_time(DateTime1)

This function is deprecated. calendar:local_time_to_universal_time/1 is deprecated; use calendar:local_time_to_universal_time_dst/1 instead.

-spec local_time_to_universal_time(DateTime1) -> DateTime2
                                      when DateTime1 :: datetime1970(), DateTime2 :: datetime1970().

Converts from local time to Universal Coordinated Time (UTC). DateTime1 must refer to a local date after Jan 1, 1970.

Warning

This function is deprecated. Use local_time_to_universal_time_dst/1 instead, as it gives a more correct and complete result. Especially for the period that does not exist, as it is skipped during the switch to daylight saving time, this function still returns a result.

local_time_to_universal_time_dst(DateTime1)

-spec local_time_to_universal_time_dst(DateTime1) -> [DateTime]
                                          when DateTime1 :: datetime1970(), DateTime :: datetime1970().

Converts from local time to Universal Coordinated Time (UTC). DateTime1 must refer to a local date after Jan 1, 1970.

The return value is a list of 0, 1, or 2 possible UTC times:

[] - For a local {Date1, Time1} during the period that is skipped when switching to daylight saving time, there is no corresponding UTC, as the local time is illegal (it has never occured).

[DstDateTimeUTC, DateTimeUTC] - For a local {Date1, Time1} during the period that is repeated when switching from daylight saving time, two corresponding UTCs exist; one for the first instance of the period when daylight saving time is still active, and one for the second instance.

[DateTimeUTC] - For all other local times only one corresponding UTC exists.

now_to_datetime(Now)

-spec now_to_datetime(Now) -> datetime1970() when Now :: erlang:timestamp().

Returns Universal Coordinated Time (UTC) converted from the return value from erlang:timestamp/0.

now_to_local_time(Now)

-spec now_to_local_time(Now) -> datetime1970() when Now :: erlang:timestamp().

Returns local date and time converted from the return value from erlang:timestamp/0.

now_to_universal_time(Now)

-spec now_to_universal_time(Now) -> datetime1970() when Now :: erlang:timestamp().

Returns Universal Coordinated Time (UTC) converted from the return value from erlang:timestamp/0.

rfc3339_to_system_time(DateTimeString)

(since OTP 21.0)

-spec rfc3339_to_system_time(DateTimeString) -> integer() when DateTimeString :: rfc3339_string().

Equivalent to rfc3339_to_system_time(DateTimeString, []).

rfc3339_to_system_time(DateTimeString, Options)

(since OTP 21.0)

-spec rfc3339_to_system_time(DateTimeString, Options) -> integer()
                                when
                                    DateTimeString :: rfc3339_string(),
                                    Options :: [Option],
                                    Option :: {unit, rfc3339_time_unit()}.

Converts an RFC 3339 timestamp into system time. The data format of RFC 3339 timestamps is described by RFC 3339. Starting from OTP 25.1, the minutes part of the time zone is optional.

Valid option:

{unit, Unit} - The time unit of the return value. The default is second.

1> calendar:rfc3339_to_system_time("2018-02-01T16:17:58+01:00").
1517498278
2> calendar:rfc3339_to_system_time("2018-02-01 15:18:02.088Z",
   [{unit, nanosecond}]).
1517498282088000000
3> calendar:rfc3339_to_system_time(<<"2018-02-01 15:18:02.088Z">>,
   [{unit, nanosecond}]).
1517498282088000000

seconds_to_daystime(Seconds)

-spec seconds_to_daystime(Seconds) -> {Days, Time}
                             when Seconds :: integer(), Days :: integer(), Time :: time().

Converts a specified number of seconds into days, hours, minutes, and seconds. Time is always non-negative, but Days is negative if argument Seconds is.

seconds_to_time(Seconds)

-spec seconds_to_time(Seconds) -> time() when Seconds :: secs_per_day().

Computes the time from the specified number of seconds. Seconds must be less than the number of seconds per day (86400).

system_time_to_local_time(Time, TimeUnit)

(since OTP 21.0)

-spec system_time_to_local_time(Time, TimeUnit) -> datetime()
                                   when Time :: integer(), TimeUnit :: erlang:time_unit().

Converts a specified system time into local date and time.

system_time_to_rfc3339(Time)

(since OTP 21.0)

-spec system_time_to_rfc3339(Time) -> DateTimeString
                                when Time :: integer(), DateTimeString :: rfc3339_string().

Equivalent to system_time_to_rfc3339(Time, []).

system_time_to_rfc3339(Time, Options)

(since OTP 21.0)

-spec system_time_to_rfc3339(Time, Options) -> DateTimeString
                                when
                                    Time :: integer(),
                                    Options :: [Option],
                                    Option ::
                                        {offset, offset()} |
                                        {time_designator, byte()} |
                                        {unit, rfc3339_time_unit()} |
                                        {return, string | binary},
                                    DateTimeString :: rfc3339_string().

Converts a system time into an RFC 3339 timestamp.

The data format of RFC 3339 timestamps is described by RFC 3339. The data format of offsets is also described by RFC 3339.

Valid options:

{offset, Offset} - The offset, either a string or an integer, to be included in the formatted string. An empty string, which is the default, is interpreted as local time. A non-empty string is included as is. The time unit of the integer is the same as the one of Time.

{time_designator, Character} - The character used as time designator, that is, the date and time separator. The default is $T.

{unit, Unit} - The time unit of Time. The default is second. If some other unit is given (millisecond, microsecond, nanosecond, or native), the formatted string includes a fraction of a second. The number of fractional second digits is three, six, or nine depending on what time unit is chosen. For native three fractional digits are included. Notice that trailing zeros are not removed from the fraction.

{return, Return} - The desired encoding type for the output, whether a string or a binary is desired. Defaults to string.

1> calendar:system_time_to_rfc3339(erlang:system_time(second)).
"2018-04-23T14:56:28+02:00"
2> calendar:system_time_to_rfc3339(erlang:system_time(second),
   [{offset, "-02:00"}]).
"2018-04-23T10:56:52-02:00"
3> calendar:system_time_to_rfc3339(erlang:system_time(second),
   [{offset, -7200}]).
"2018-04-23T10:57:05-02:00"
4> calendar:system_time_to_rfc3339(erlang:system_time(millisecond),
   [{unit, millisecond}, {time_designator, $\s}, {offset, "Z"}]).
"2018-04-23 12:57:20.482Z"
5> calendar:system_time_to_rfc3339(erlang:system_time(millisecond),
   [{unit, millisecond}, {time_designator, $\s}, {offset, "Z"}, {return, binary}]).
<<"2018-04-23 12:57:20.482Z">>

system_time_to_universal_time(Time, TimeUnit)

(since OTP 21.0)

-spec system_time_to_universal_time(Time, TimeUnit) -> datetime()
                                       when Time :: integer(), TimeUnit :: erlang:time_unit().

Converts a specified system time into universal date and time.

time_difference(T1, T2)

-spec time_difference(T1, T2) -> {Days, Time}
                         when T1 :: datetime(), T2 :: datetime(), Days :: integer(), Time :: time().

Returns the difference between two {Date, Time} tuples. T2 is to refer to an epoch later than T1.

Warning

This function is obsolete. Use the conversion functions for gregorian days and seconds instead.

time_to_seconds(Time)

-spec time_to_seconds(Time) -> secs_per_day() when Time :: time().

Returns the number of seconds since midnight up to the specified time.

universal_time()

-spec universal_time() -> datetime().

Returns the Universal Coordinated Time (UTC) reported by the underlying operating system. Returns local time if universal time is unavailable.

universal_time_to_local_time(DateTime)

-spec universal_time_to_local_time(DateTime) -> datetime() when DateTime :: datetime1970().

Converts from Universal Coordinated Time (UTC) to local time. DateTime must refer to a date after Jan 1, 1970.

universal_time_to_system_time(UniversalTime)

(since OTP 28.0)

-spec universal_time_to_system_time(datetime()) -> integer().

Equivalent to universal_time_to_system_time(LocalTime, []).

universal_time_to_system_time(DateTime, Options)

(since OTP 28.0)

-spec universal_time_to_system_time(datetime(), Options) -> integer()
                                       when Options :: [Option], Option :: {unit, erlang:time_unit()}.

Converts universal time into system time.

valid_date(Date)

-spec valid_date(Date) -> boolean() when Date :: date().

This function checks if a date is a valid.

valid_date(Year, Month, Day)

-spec valid_date(Year, Month, Day) -> boolean()
                    when Year :: integer(), Month :: integer(), Day :: integer().

Equivalent to valid_date({Year, Month, Day}).

timer

Timer functions.

This module provides useful functions related to time. Unless otherwise stated, time is always measured in milliseconds. All timer functions return immediately, regardless of work done by another process.

Successful evaluations of the timer functions give return values containing a timer reference, denoted TRef. By using cancel/1, the returned reference can be used to cancel any requested action. A TRef is an Erlang term, which contents must not be changed.

The time-outs are not exact, but are at least as long as requested.

Creating timers using erlang:send_after/3 and erlang:start_timer/3 is more efficient than using the timers provided by this module. However, the timer module has been improved in OTP 25, making it more efficient and less susceptible to being overloaded. See the Timer Module section in the Efficiency Guide.

For more information on timers in Erlang in general, see the Timers section of the Time and Time Correction in Erlang ERTS User's guide.

Examples

Example 1

The following example shows how to print "Hello World!" in 5 seconds:

1> timer:apply_after(5000, io, format, ["~nHello World!~n", []]).
{ok,TRef}
Hello World!

Example 2

The following example shows a process performing a certain action, and if this action is not completed within a certain limit, the process is killed:

Pid = spawn(mod, fun, [foo, bar]),
%% If pid is not finished in 10 seconds, kill him
{ok, R} = timer:kill_after(timer:seconds(10), Pid),
...
%% We change our mind...
timer:cancel(R),
...

Notes

A timer can always be removed by calling cancel/1.

An interval timer, that is, a timer created by evaluating any of the functions apply_interval/2, apply_interval/3, apply_interval/4, apply_repeatedly/2, apply_repeatedly/3, apply_repeatedly/4, send_interval/2, and send_interval/3 is linked to the process to which the timer performs its task.

A one-shot timer, that is, a timer created by evaluating any of the functions apply_after/2, apply_after/3, apply_after/4, send_after/2, send_after/3, exit_after/2, exit_after/3, kill_after/1, and kill_after/2 is not linked to any process. Hence, such a timer is removed only when it reaches its time-out, or if it is explicitly removed by a call to cancel/1.

The functions given to apply_after/2, apply_after/3, apply_interval/2, apply_interval/3, apply_repeatedly/2, and apply_repeatedly/3, or denoted by Module, Function and Arguments given to apply_after/4, apply_interval/4, and apply_repeatedly/4 are executed in a freshly-spawned process, and therefore calls to self/0 in those functions will return the Pid of this process, which is different from the process that called timer:apply_*.

Example

In the following example, the intention is to set a timer to execute a function after 1 second, which performs a fictional task, and then wants to inform the process which set the timer about its completion, by sending it a done message.

Using self/0 inside the timed function, the code below does not work as intended. The task gets done, but the done message gets sent to the wrong process and is lost.

1> timer:apply_after(1000, fun() -> do_something(), self() ! done end).
{ok,TRef}
2> receive done -> done after 5000 -> timeout end.
%% ... 5s pass...
timeout

The code below calls self/0 in the process which sets the timer and assigns it to a variable, which is then used in the function to send the done message to, and so works as intended.

1> Target = self()
<0.82.0>
2> timer:apply_after(1000, fun() -> do_something(), Target ! done end).
{ok,TRef}
3> receive done -> done after 5000 -> timeout end.
%% ... 1s passes...
done

Another option is to pass the message target as a parameter to the function.

1> timer:apply_after(1000, fun(Target) -> do_something(), Target ! done end, [self()]).
{ok,TRef}
2> receive done -> done after 5000 -> timeout end.
%% ... 1s passes...
done

Summary

Types

time()

Time in milliseconds.

tref()

A timer reference.

Functions

apply_after(Time, Function)

Evaluates spawn(erlang, apply, [Function, []]) after Time milliseconds.

apply_after(Time, Function, Arguments)

Evaluates spawn(erlang, apply, [Function, Arguments]) after Time milliseconds.

apply_after(Time, Module, Function, Arguments)

Evaluates spawn(Module, Function, Arguments) after Time milliseconds.

apply_interval(Time, Function)

Evaluates spawn(erlang, apply, [Function, []]) repeatedly at intervals of Time, irrespective of whether a previously spawned process has finished or not.

apply_interval(Time, Function, Arguments)

Evaluates spawn(erlang, apply, [Function, Arguments]) repeatedly at intervals of Time, irrespective of whether a previously spawned process has finished or not.

apply_interval(Time, Module, Function, Arguments)

Evaluates spawn(Module, Function, Arguments) repeatedly at intervals of Time, irrespective of whether a previously spawned process has finished or not.

apply_repeatedly(Time, Function)

Evaluates spawn(erlang, apply, [Function, []]) repeatedly at intervals of Time, waiting for the spawned process to finish before starting the next.

apply_repeatedly(Time, Function, Arguments)

Evaluates spawn(erlang, apply, [Function, Arguments]) repeatedly at intervals of Time, waiting for the spawned process to finish before starting the next.

apply_repeatedly(Time, Module, Function, Arguments)

Evaluates spawn(Module, Function, Arguments) repeatedly at intervals of Time, waiting for the spawned process to finish before starting the next.

cancel(TRef)

Cancels a previously requested time-out. TRef is a unique timer reference returned by the related timer function.

exit_after(Time, Reason1)

Equivalent to exit_after(Time, self(), Reason).

exit_after(Time, Target, Reason1)

Sends an exit signal with reason Reason1 to Target, which can be a local process identifier or an atom of a registered name.

hms(Hours, Minutes, Seconds)

Returns the number of milliseconds in Hours + Minutes + Seconds.

hours(Hours)

Returns the number of milliseconds in Hours.

kill_after(Time)

Equivalent to exit_after(Time, self(), kill).

kill_after(Time, Target)

Equivalent to exit_after(Time, Target, kill).

minutes(Minutes)

Returns the number of milliseconds in Minutes.

now_diff(T2, T1)

Calculates the time difference Tdiff = T2 - T1 in microseconds, where T1 and T2 are time-stamp tuples on the same format as returned from erlang:timestamp/0 or os:timestamp/0.

seconds(Seconds)

Returns the number of milliseconds in Seconds.

send_after(Time, Message)

Equivalent to send_after(Time, self(), Message).

send_after(Time, Destination, Message)

Evaluates Destination ! Message after Time milliseconds.

send_interval(Time, Message)

Equivalent to send_interval(Time, self(), Message).

send_interval(Time, Destination, Message)

Evaluates Destination ! Message repeatedly after Time milliseconds.

sleep(Time)

Suspends the process calling this function for Time milliseconds and then returns ok, or suspends the process forever if Time is the atom infinity. Naturally, this function does not return immediately.

start()

Starts the timer server.

tc(Fun)

Equivalent to tc(Fun, microsecond).

tc(Fun, ArgumentsOrTimeUnit)

Measures the execution time of Fun.

tc(ModuleOrFun, FunctionOrArguments, ArgumentsOrTimeUnit)

Measures the execution time of Fun or apply(Module, Function, Arguments).

tc(Module, Function, Arguments, TimeUnit)

Evaluates apply(Module, Function, Arguments) and measures the elapsed real time as reported by erlang:monotonic_time/0.

Types

time()

(since OTP 28.0)

-nominal time() :: non_neg_integer().

Time in milliseconds.

tref()

-opaque tref()

A timer reference.

Functions

apply_after(Time, Function)

(since OTP 27.0)

-spec apply_after(Time, Function) -> {ok, TRef} | {error, Reason}
                     when Time :: time(), Function :: fun(() -> _), TRef :: tref(), Reason :: term().

Evaluates spawn(erlang, apply, [Function, []]) after Time milliseconds.

apply_after(Time, Function, Arguments)

(since OTP 27.0)

-spec apply_after(Time, Function, Arguments) -> {ok, TRef} | {error, Reason}
                     when
                         Time :: time(),
                         Function :: fun((...) -> _),
                         Arguments :: [term()],
                         TRef :: tref(),
                         Reason :: term().

Evaluates spawn(erlang, apply, [Function, Arguments]) after Time milliseconds.

apply_after(Time, Module, Function, Arguments)

-spec apply_after(Time, Module, Function, Arguments) -> {ok, TRef} | {error, Reason}
                     when
                         Time :: time(),
                         Module :: module(),
                         Function :: atom(),
                         Arguments :: [term()],
                         TRef :: tref(),
                         Reason :: term().

Evaluates spawn(Module, Function, Arguments) after Time milliseconds.

apply_interval(Time, Function)

(since OTP 27.0)

-spec apply_interval(Time, Function) -> {ok, TRef} | {error, Reason}
                        when Time :: time(), Function :: fun(() -> _), TRef :: tref(), Reason :: term().

Evaluates spawn(erlang, apply, [Function, []]) repeatedly at intervals of Time, irrespective of whether a previously spawned process has finished or not.

apply_interval(Time, Function, Arguments)

(since OTP 27.0)

-spec apply_interval(Time, Function, Arguments) -> {ok, TRef} | {error, Reason}
                        when
                            Time :: time(),
                            Function :: fun((...) -> _),
                            Arguments :: [term()],
                            TRef :: tref(),
                            Reason :: term().

Evaluates spawn(erlang, apply, [Function, Arguments]) repeatedly at intervals of Time, irrespective of whether a previously spawned process has finished or not.

apply_interval(Time, Module, Function, Arguments)

-spec apply_interval(Time, Module, Function, Arguments) -> {ok, TRef} | {error, Reason}
                        when
                            Time :: time(),
                            Module :: module(),
                            Function :: atom(),
                            Arguments :: [term()],
                            TRef :: tref(),
                            Reason :: term().

Evaluates spawn(Module, Function, Arguments) repeatedly at intervals of Time, irrespective of whether a previously spawned process has finished or not.

Warning

If the execution time of the spawned process is, on average, greater than the given Time, multiple such processes will run at the same time. With long execution times, short intervals, and many interval timers running, this may even lead to exceeding the number of allowed processes. As an extreme example, consider [timer:apply_interval(1, timer, sleep, [1000]) || _ <- lists:seq(1, 1000)], that is, 1,000 interval timers executing a process that takes 1s to complete, started in intervals of 1ms, which would result in 1,000,000 processes running at the same time, far more than a node started with default settings allows (see the System Limits section in the Effiency Guide).

apply_repeatedly(Time, Function)

(since OTP 27.0)

-spec apply_repeatedly(Time, Function) -> {ok, TRef} | {error, Reason}
                          when
                              Time :: time(), Function :: fun(() -> _), TRef :: tref(), Reason :: term().

Evaluates spawn(erlang, apply, [Function, []]) repeatedly at intervals of Time, waiting for the spawned process to finish before starting the next.

apply_repeatedly(Time, Function, Arguments)

(since OTP 27.0)

-spec apply_repeatedly(Time, Function, Arguments) -> {ok, TRef} | {error, Reason}
                          when
                              Time :: time(),
                              Function :: fun((...) -> _),
                              Arguments :: [term()],
                              TRef :: tref(),
                              Reason :: term().

Evaluates spawn(erlang, apply, [Function, Arguments]) repeatedly at intervals of Time, waiting for the spawned process to finish before starting the next.

apply_repeatedly(Time, Module, Function, Arguments)

(since OTP 26.0)

-spec apply_repeatedly(Time, Module, Function, Arguments) -> {ok, TRef} | {error, Reason}
                          when
                              Time :: time(),
                              Module :: module(),
                              Function :: atom(),
                              Arguments :: [term()],
                              TRef :: tref(),
                              Reason :: term().

Evaluates spawn(Module, Function, Arguments) repeatedly at intervals of Time, waiting for the spawned process to finish before starting the next.

If the execution time of the spawned process is greater than the given Time, the next process is spawned immediately after the one currently running has finished. Assuming that execution times of the spawned processes performing the applies on average are smaller than Time, the amount of applies made over a large amount of time will be the same even if some individual execution times are larger than Time. The system will try to catch up as soon as possible. For example, if one apply takes 2.5*Time, the following two applies will be made immediately one after the other in sequence.

cancel(TRef)

-spec cancel(TRef) -> {ok, cancel} | {error, Reason} when TRef :: tref(), Reason :: term().

Cancels a previously requested time-out. TRef is a unique timer reference returned by the related timer function.

Returns {ok, cancel}, or {error, Reason} when TRef is not a timer reference.

exit_after(Time, Reason1)

-spec exit_after(Time, Reason1) -> {ok, TRef} | {error, Reason2}
                    when Time :: time(), TRef :: tref(), Reason1 :: term(), Reason2 :: term().

Equivalent to exit_after(Time, self(), Reason).

exit_after(Time, Target, Reason1)

-spec exit_after(Time, Target, Reason1) -> {ok, TRef} | {error, Reason2}
                    when
                        Time :: time(),
                        Target :: pid() | (RegName :: atom()),
                        TRef :: tref(),
                        Reason1 :: term(),
                        Reason2 :: term().

Sends an exit signal with reason Reason1 to Target, which can be a local process identifier or an atom of a registered name.

hms(Hours, Minutes, Seconds)

-spec hms(Hours, Minutes, Seconds) -> MilliSeconds
             when
                 Hours :: non_neg_integer(),
                 Minutes :: non_neg_integer(),
                 Seconds :: non_neg_integer(),
                 MilliSeconds :: time().

Returns the number of milliseconds in Hours + Minutes + Seconds.

hours(Hours)

-spec hours(Hours) -> MilliSeconds when Hours :: non_neg_integer(), MilliSeconds :: time().

Returns the number of milliseconds in Hours.

kill_after(Time)

-spec kill_after(Time) -> {ok, TRef} | {error, Reason2}
                    when Time :: time(), TRef :: tref(), Reason2 :: term().

Equivalent to exit_after(Time, self(), kill).

kill_after(Time, Target)

-spec kill_after(Time, Target) -> {ok, TRef} | {error, Reason2}
                    when
                        Time :: time(),
                        Target :: pid() | (RegName :: atom()),
                        TRef :: tref(),
                        Reason2 :: term().

Equivalent to exit_after(Time, Target, kill).

minutes(Minutes)

-spec minutes(Minutes) -> MilliSeconds when Minutes :: non_neg_integer(), MilliSeconds :: time().

Returns the number of milliseconds in Minutes.

now_diff(T2, T1)

-spec now_diff(T2, T1) -> Tdiff
                  when T1 :: erlang:timestamp(), T2 :: erlang:timestamp(), Tdiff :: integer().

Calculates the time difference Tdiff = T2 - T1 in microseconds, where T1 and T2 are time-stamp tuples on the same format as returned from erlang:timestamp/0 or os:timestamp/0.

seconds(Seconds)

-spec seconds(Seconds) -> MilliSeconds when Seconds :: non_neg_integer(), MilliSeconds :: time().

Returns the number of milliseconds in Seconds.

send_after(Time, Message)

-spec send_after(Time, Message) -> {ok, TRef} | {error, Reason}
                    when Time :: time(), Message :: term(), TRef :: tref(), Reason :: term().

Equivalent to send_after(Time, self(), Message).

send_after(Time, Destination, Message)

-spec send_after(Time, Destination, Message) -> {ok, TRef} | {error, Reason}
                    when
                        Time :: time(),
                        Destination :: pid() | (RegName :: atom()) | {RegName :: atom(), Node :: node()},
                        Message :: term(),
                        TRef :: tref(),
                        Reason :: term().

Evaluates Destination ! Message after Time milliseconds.

Destination can be a remote or local process identifier, an atom of a registered name or a tuple {RegName, Node} for a registered name at another node.

See also the Timer Module section in the Efficiency Guide.

send_interval(Time, Message)

-spec send_interval(Time, Message) -> {ok, TRef} | {error, Reason}
                       when Time :: time(), Message :: term(), TRef :: tref(), Reason :: term().

Equivalent to send_interval(Time, self(), Message).

send_interval(Time, Destination, Message)

-spec send_interval(Time, Destination, Message) -> {ok, TRef} | {error, Reason}
                       when
                           Time :: time(),
                           Destination ::
                               pid() | (RegName :: atom()) | {RegName :: atom(), Node :: node()},
                           Message :: term(),
                           TRef :: tref(),
                           Reason :: term().

Evaluates Destination ! Message repeatedly after Time milliseconds.

Destination can be a remote or local process identifier, an atom of a registered name or a tuple {RegName, Node} for a registered name at another node.

sleep(Time)

-spec sleep(Time) -> ok when Time :: time() | infinity.

Suspends the process calling this function for Time milliseconds and then returns ok, or suspends the process forever if Time is the atom infinity. Naturally, this function does not return immediately.

Note

Before OTP 25, timer:sleep/1 did not accept integer timeout values greater than 16#ffffffff, that is, 2^32-1. Since OTP 25, arbitrarily high integer values are accepted.

start()

-spec start() -> ok.

Starts the timer server.

Normally, the server does not need to be started explicitly. It is started dynamically if it is needed. This is useful during development, but in a target system the server is to be started explicitly. Use configuration parameters for Kernel for this.

tc(Fun)

(since OTP R14B03)

-spec tc(Fun) -> {Time, Value} when Fun :: function(), Time :: integer(), Value :: term().

Equivalent to tc(Fun, microsecond).

tc(Fun, ArgumentsOrTimeUnit)

(since OTP R14B)

-spec tc(Fun, Arguments) -> {Time, Value}
            when Fun :: function(), Arguments :: [term()], Time :: integer(), Value :: term();
        (Fun, TimeUnit) -> {Time, Value}
            when Fun :: function(), TimeUnit :: erlang:time_unit(), Time :: integer(), Value :: term().

Measures the execution time of Fun.

Equivalent to tc(Fun, Arguments, microsecond) if called as tc(Fun, Arguments).

Measures the execution time of Fun in TimeUnit if called as tc(Fun, TimeUnit). Added in OTP 26.0.

tc(ModuleOrFun, FunctionOrArguments, ArgumentsOrTimeUnit)

-spec tc(Module, Function, Arguments) -> {Time, Value}
            when
                Module :: module(),
                Function :: atom(),
                Arguments :: [term()],
                Time :: integer(),
                Value :: term();
        (Fun, Arguments, TimeUnit) -> {Time, Value}
            when
                Fun :: function(),
                Arguments :: [term()],
                TimeUnit :: erlang:time_unit(),
                Time :: integer(),
                Value :: term().

Measures the execution time of Fun or apply(Module, Function, Arguments).

Equivalent to tc(Module, Function, Arguments, microsecond) if called as tc(Module, Function, Arguments).

Equivalent to tc(erlang, apply, [Fun, Arguments], TimeUnit) if called as tc(Fun, Arguments, TimeUnit). Added in OTP 26.0

tc(Module, Function, Arguments, TimeUnit)

(since OTP 26.0)

-spec tc(Module, Function, Arguments, TimeUnit) -> {Time, Value}
            when
                Module :: module(),
                Function :: atom(),
                Arguments :: [term()],
                TimeUnit :: erlang:time_unit(),
                Time :: integer(),
                Value :: term().

Evaluates apply(Module, Function, Arguments) and measures the elapsed real time as reported by erlang:monotonic_time/0.

Returns {Time, Value}, where Time is the elapsed real time in the specified TimeUnit, and Value is what is returned from the apply.

argparse

Command line arguments parser.

This module implements command line parser. Parser operates with commands and arguments represented as a tree. Commands are branches, and arguments are leaves of the tree. Parser always starts with the root command, named after progname (the name of the program which started Erlang).

A command specification may contain handler definition for each command, and a number argument specifications. When parser is successful, argparse calls the matching handler, passing arguments extracted from the command line. Arguments can be positional (occupying specific position in the command line), and optional, residing anywhere but prefixed with a specified character.

argparse automatically generates help and usage messages. It will also issue errors when users give the program invalid arguments.

Quick start

argparse is designed to work with escript. The example below is a fully functioning Erlang program accepting two command line arguments and printing their product.

#!/usr/bin/env escript

main(Args) ->
    argparse:run(Args, cli(), #{progname => mul}).

cli() ->
    #{
        arguments => [
            #{name => left, type => integer},
            #{name => right, type => integer}
        ],
        handler =>
            fun (#{left := Left, right := Right}) ->
                io:format("~b~n", [Left * Right])
            end
    }.

Running this script with no arguments results in an error, accompanied by the usage information.

The cli function defines a single command with embedded handler accepting a map. Keys of the map are argument names as defined by the argument field of the command, left and right in the example. Values are taken from the command line, and converted into integers, as requested by the type specification. Both arguments in the example above are required (and therefore defined as positional).

Command hierarchy

A command may contain nested commands, forming a hierarchy. Arguments defined at the upper level command are automatically added to all nested commands. Nested commands example (assuming progname is nested):

cli() ->
  #{
    %% top level argument applicable to all commands
    arguments => [#{name => top}],
      commands => #{
        "first" => #{
          %% argument applicable to "first" command and
          %%  all commands nested into "first"
          arguments => [#{name => mid}],
          commands => #{
            "second" => #{
              %% argument only applicable for "second" command
              arguments => [#{name => bottom}],
              handler => fun (A) -> io:format("~p~n", [A]) end
          }
        }
      }
    }
  }.

In the example above, a 3-level hierarchy is defined. First is the script itself (nested), accepting the only argument top. Since it has no associated handler, run/3 will not accept user input omitting nested command selection. For this example, user has to supply 5 arguments in the command line, two being command names, and another 3 - required positional arguments:

./nested.erl one first second two three
#{top => "one",mid => "two",bottom => "three"}

Commands have preference over positional argument values. In the example above, commands and positional arguments are interleaving, and argparse matches command name first.

Arguments

argparse supports positional and optional arguments. Optional arguments, or options for short, must be prefixed with a special character (- is the default on all operating systems). Both options and positional arguments have 1 or more associated values. See argument specification to find more details about supported combinations.

In the user input, short options may be concatenated with their values. Long options support values separated by =. Consider this definition:

cli() ->
  #{
    arguments => [
      #{name => long, long => "-long"},
      #{name => short, short => $s}
    ],
    handler => fun (Args) -> io:format("~p~n", [Args]) end
  }.

Running ./args --long=VALUE prints #{long => "VALUE"}, running ./args -sVALUE prints #{short => "VALUE"}

argparse supports boolean flags concatenation: it is possible to shorten -r -f -v to -rfv.

Shortened option names are not supported: it is not possible to use --my-argum instead of --my-argument-name even when such option can be unambiguously found.

Summary

Types

arg_map()

Arguments map is the map of argument names to the values extracted from the command line. It is passed to the matching command handler. If an argument is omitted, but has the default value is specified, it is added to the map. When no default value specified, and argument is not present in the command line, corresponding key is not present in the resulting map.

arg_type()

Defines type conversion applied to the string retrieved from the user input. If the conversion is successful, resulting value is validated using optional Choices, or minimums and maximums (for integer and floating point values only). Strings and binary values may be validated using regular expressions. It's possible to define custom type conversion function, accepting a string and returning Erlang term. If this function raises error with badarg reason, argument is treated as invalid.

args()

List of command line arguments to be parsed.

argument()

Argument specification. Defines a single named argument that is returned in the argument map. The only required field is name, all other fields have defaults.

argument_help()

User-defined help template to print in the command usage. First element of a tuple must be a string. It is printed as a part of the usage header. Second element of the tuple can be either a list containing strings, type and default atoms, or a user-defined function that must return a string. A plain string should be wrapped as a list such as ["string is nested"].

argument_name()

Argument name is used to populate argument map.

cmd_path()

Path to the nested command. First element is always the progname, subsequent elements are nested command names.

command()

Command specification. May contain nested commands, forming a hierarchy.

command_help()

User-defined help template. Use this option to mix custom and predefined usage text. Help template may contain unicode strings, and following atoms

handler()

Command handler specification. Called by run/3 upon successful parser return.

parse_result()

Returned from parse/2,3. Contains arguments extracted from the command line, path to the nested command (if any), and a (potentially nested) command specification that was considered when the parser finished successfully. It is expected that the command contains a handler definition, that will be called passing the argument map.

parser_error()

Returned from parse/2,3 when the user input cannot be parsed according to the command specification.

parser_options()

Options changing parser behaviour.

Functions

format_error(Reason)

Generates human-readable text for parser error. Does not include help/usage information, and does not provide localisation.

help(Command)

Equivalent to help/2.

help(Command, Options)

Generates help/usage information text for the command supplied, or any nested command when command option is specified. Arguments are displayed in the same order as specified in Command. Does not provide localisation. Expects progname to be set, otherwise defaults to return value of init:get_argument(progname).

parse(Args, Command)

Equivalent to parse/3.

parse(Args, Command, Options)

Parses command line arguments according to the command specification. Raises an exception if the command specification is not valid. Use erl_error:format_exception/3,4 to see a friendlier message. Invalid command line input does not raise an exception, but makes parse/2,3 to return a tuple {error, parser_error()}.

run(Args, Command, Options)

Parses command line arguments and calls the matching command handler. Prints human-readable error, help/usage information for the discovered command, and halts the emulator with code 1 if there is any error in the command specification or user-provided command line input.

Types

arg_map()

(since OTP 26.0)

-type arg_map() :: #{argument_name() => term()}.

Arguments map is the map of argument names to the values extracted from the command line. It is passed to the matching command handler. If an argument is omitted, but has the default value is specified, it is added to the map. When no default value specified, and argument is not present in the command line, corresponding key is not present in the resulting map.

arg_type()

(since OTP 26.0)

-type arg_type() ::
          boolean | float |
          {float, Choice :: [float()]} |
          {float, [{min, float()} | {max, float()}]} |
          integer |
          {integer, Choices :: [integer()]} |
          {integer, [{min, integer()} | {max, integer()}]} |
          string |
          {string, Choices :: [string()]} |
          {string, Re :: string()} |
          {string, Re :: string(), ReOptions :: [term()]} |
          binary |
          {binary, Choices :: [binary()]} |
          {binary, Re :: binary()} |
          {binary, Re :: binary(), ReOptions :: [term()]} |
          atom |
          {atom, Choices :: [atom()]} |
          {atom, unsafe} |
          {custom, fun((string()) -> term())}.

Defines type conversion applied to the string retrieved from the user input. If the conversion is successful, resulting value is validated using optional Choices, or minimums and maximums (for integer and floating point values only). Strings and binary values may be validated using regular expressions. It's possible to define custom type conversion function, accepting a string and returning Erlang term. If this function raises error with badarg reason, argument is treated as invalid.

args()

(since OTP 26.0)

-type args() :: [string() | unicode:chardata()].

List of command line arguments to be parsed.

argument()

(since OTP 26.0)

-type argument() ::
          #{name := argument_name(),
            short => char(),
            long => string(),
            required => boolean(),
            default => term(),
            type => arg_type(),
            action => store | {store, term()} | append | {append, term()} | count | extend,
            nargs => pos_integer() | 'maybe' | {'maybe', term()} | list | nonempty_list | all,
            help => hidden | unicode:chardata() | argument_help()}.

Argument specification. Defines a single named argument that is returned in the argument map. The only required field is name, all other fields have defaults.

If either of the short or long fields is specified, the argument is treated as optional. Optional arguments do not have specific order and may appear anywhere in the command line. Positional arguments are ordered the same way as they appear in the arguments list of the command specification.

By default, all positional arguments must be present in the command line. The parser will return an error otherwise. Options, however, may be omitted, in which case resulting argument map will either contain the default value, or not have the key at all.

name - Sets the argument name in the parsed argument map. If help is not defined, name is also used to generate the default usage message.

short - Defines a short (single character) form of an optional argument.

%% Define a command accepting argument named myarg, with short form $a:
1> Cmd = #{arguments => [#{name => myarg, short => $a}]}.
%% Parse command line "-a str":
2> {ok, ArgMap, _, _} = argparse:parse(["-a", "str"], Cmd), ArgMap.

#{myarg => "str"}

%% Option value can be concatenated with the switch: "-astr"
3> {ok, ArgMap, _, _} = argparse:parse(["-astr"], Cmd), ArgMap.

#{myarg => "str"}

By default all options expect a single value following the option switch. The only exception is an option of a boolean type.

long - Defines a long form of an optional argument.

1> Cmd = #{arguments => [#{name => myarg, long => "name"}]}.
%% Parse command line "-name Erlang":
2> {ok, ArgMap, _, _} = argparse:parse(["-name", "Erlang"], Cmd), ArgMap.

#{myarg => "Erlang"}
%% Or use "=" to separate the switch and the value:
3> {ok, ArgMap, _, _} = argparse:parse(["-name=Erlang"], Cmd), ArgMap.

#{myarg => "Erlang"}

If neither short not long is defined, the argument is treated as positional.

required - Forces the parser to expect the argument to be present in the command line. By default, all positional argument are required, and all options are not.

default - Specifies the default value to put in the parsed argument map if the value is not supplied in the command line.

1> argparse:parse([], #{arguments => [#{name => myarg, short => $m}]}).

{ok,#{}, ...
2> argparse:parse([], #{arguments => [#{name => myarg, short => $m, default => "def"}]}).

{ok,#{myarg => "def"}, ...

type - Defines type conversion and validation routine. The default is string, assuming no conversion.

nargs - Defines the number of following arguments to consume from the command line. By default, the parser consumes the next argument and converts it into an Erlang term according to the specified type.

pos_integer/0 - Consume exactly this number of positional arguments, fail if there is not enough. Value in the argument map contains a list of exactly this length. Example, defining a positional argument expecting 3 integer values:

1> Cmd = #{arguments => [#{name => ints, type => integer, nargs => 3}]},
argparse:parse(["1", "2", "3"], Cmd).

{ok, #{ints => [1, 2, 3]}, ...

Another example defining an option accepted as -env and expecting two string arguments:

1> Cmd = #{arguments => [#{name => env, long => "env", nargs => 2}]},
argparse:parse(["-env", "key", "value"], Cmd).

{ok, #{env => ["key", "value"]}, ...

list - Consume all following arguments until hitting the next option (starting with an option prefix). May result in an empty list added to the arguments map.

1> Cmd = #{arguments => [
  #{name => nodes, long => "nodes", nargs => list},
  #{name => verbose, short => $v, type => boolean}
]},
argparse:parse(["-nodes", "one", "two", "-v"], Cmd).

{ok, #{nodes => ["one", "two"], verbose => true}, ...

nonempty_list - Same as list, but expects at least one argument. Returns an error if the following command line argument is an option switch (starting with the prefix).

'maybe' - Consumes the next argument from the command line, if it does not start with an option prefix. Otherwise, adds a default value to the arguments map.

1> Cmd = #{arguments => [
  #{name => level, short => $l, nargs => 'maybe', default => "error"},
  #{name => verbose, short => $v, type => boolean}
]},
argparse:parse(["-l", "info", "-v"], Cmd).

{ok,#{level => "info",verbose => true}, ...

%% When "info" is omitted, argument maps receives the default "error"
2> argparse:parse(["-l", "-v"], Cmd).

{ok,#{level => "error",verbose => true}, ...

{'maybe', term()} - Consumes the next argument from the command line, if it does not start with an option prefix. Otherwise, adds a specified Erlang term to the arguments map.

all - Fold all remaining command line arguments into a list, ignoring any option prefixes or switches. Useful for proxying arguments into another command line utility.

1> Cmd = #{arguments => [
    #{name => verbose, short => $v, type => boolean},
    #{name => raw, long => "-", nargs => all}
]},
argparse:parse(["-v", "--", "-kernel", "arg", "opt"], Cmd).

{ok,#{raw => ["-kernel","arg","opt"],verbose => true}, ...

action - Defines an action to take when the argument is found in the command line. The default action is store.

store - Store the value in the arguments map. Overwrites the value previously written.

1> Cmd = #{arguments => [#{name => str, short => $s}]},
argparse:parse(["-s", "one", "-s", "two"], Cmd).

{ok, #{str => "two"}, ...

{store, term()} - Stores the specified term instead of reading the value from the command line.

1> Cmd = #{arguments => [#{name => str, short => $s, action => {store, "two"}}]},
argparse:parse(["-s"], Cmd).

{ok, #{str => "two"}, ...

append - Appends the repeating occurrences of the argument instead of overwriting.

1> Cmd = #{arguments => [#{name => node, short => $n, action => append}]},
argparse:parse(["-n", "one", "-n", "two", "-n", "three"], Cmd).

{ok, #{node => ["one", "two", "three"]}, ...

%% Always produces a list - even if there is one occurrence
2> argparse:parse(["-n", "one"], Cmd).

{ok, #{node => ["one"]}, ...

{append, term()} - Same as append, but instead of consuming the argument from the command line, appends a provided term/0.

count - Puts a counter as a value in the arguments map. Useful for implementing verbosity option:

1> Cmd = #{arguments => [#{name => verbose, short => $v, action => count}]},
argparse:parse(["-v"], Cmd).

{ok, #{verbose => 1}, ...

2> argparse:parse(["-vvvv"], Cmd).

{ok, #{verbose => 4}, ...

extend - Works as append, but flattens the resulting list. Valid only for nargs set to list, nonempty_list, all or pos_integer/0.

1> Cmd = #{arguments => [#{name => duet, short => $d, nargs => 2, action => extend}]},
argparse:parse(["-d", "a", "b", "-d", "c", "d"], Cmd).

{ok, #{duet => ["a", "b", "c", "d"]}, ...

%% 'append' would result in {ok, #{duet => [["a", "b"],["c", "d"]]},

help - Specifies help/usage text for the argument. argparse provides automatic generation based on the argument name, type and default value, but for better usability it is recommended to have a proper description. Setting this field to hidden suppresses usage output for this argument.

argument_help()

(since OTP 26.0)

-type argument_help() ::
          {unicode:chardata(), [unicode:chardata() | type | default] | fun(() -> unicode:chardata())}.

User-defined help template to print in the command usage. First element of a tuple must be a string. It is printed as a part of the usage header. Second element of the tuple can be either a list containing strings, type and default atoms, or a user-defined function that must return a string. A plain string should be wrapped as a list such as ["string is nested"].

argument_name()

(not exported) (since OTP 26.0)

-type argument_name() :: atom() | string() | binary().

Argument name is used to populate argument map.

cmd_path()

(since OTP 26.0)

-type cmd_path() :: [string()].

Path to the nested command. First element is always the progname, subsequent elements are nested command names.

command()

(since OTP 26.0)

-type command() ::
          #{commands => #{string() => command()},
            arguments => [argument()],
            help => hidden | unicode:chardata() | command_help(),
            handler => handler()}.

Command specification. May contain nested commands, forming a hierarchy.

commands - Maps of nested commands. Keys must be strings, matching command line input. Basic utilities do not need to specify any nested commands.

arguments - List of arguments accepted by this command, and all nested commands in the hierarchy.

help - Specifies help/usage text for this command. Pass hidden to remove this command from the usage output.

handler - Specifies a callback function to call by run/3 when the parser is successful.

command_help()

(not exported) (since OTP 26.0)

-type command_help() :: [unicode:chardata() | usage | commands | arguments | options].

User-defined help template. Use this option to mix custom and predefined usage text. Help template may contain unicode strings, and following atoms:

usage - Formatted command line usage text, e.g. rm [-rf] <directory>.

commands - Expanded list of sub-commands.

arguments - Detailed description of positional arguments.

options - Detailed description of optional arguments.

handler()

(since OTP 26.0)

-type handler() ::
          optional |
          fun((arg_map()) -> term()) |
          {module(), Fn :: atom()} |
          {fun(() -> term()), term()} |
          {module(), atom(), term()}.

Command handler specification. Called by run/3 upon successful parser return.

fun((arg_map()) -> term()) - Function accepting argument map. See the basic example in the Quick Start section.

{Module :: module(), Function :: atom()} - Function named Function, exported from Module, accepting argument map.

{fun(() -> term()), Default :: term()} - Function accepting as many arguments as there are in the arguments list for this command. Arguments missing from the parsed map are replaced with the Default. Convenient way to expose existing functions.

1> Cmd = #{arguments => [
        #{name => x, type => float},
        #{name => y, type => float, short => $p}],
    handler => {fun math:pow/2, 1}},
argparse:run(["2", "-p", "3"], Cmd, #{}).

8.0

%% default term 1 is passed to math:pow/2
2> argparse:run(["2"], Cmd, #{}).

2.0

{Module :: module(), Function :: atom(), Default :: term()} - Function named Function, exported from Module, accepting as many arguments as defined for this command. Arguments missing from the parsed map are replaced with the Default. Effectively, just a different syntax to the same functionality as demonstrated in the code above.

parse_result()

(not exported) (since OTP 26.0)

-type parse_result() :: {ok, arg_map(), Path :: cmd_path(), command()} | {error, parser_error()}.

Returned from parse/2,3. Contains arguments extracted from the command line, path to the nested command (if any), and a (potentially nested) command specification that was considered when the parser finished successfully. It is expected that the command contains a handler definition, that will be called passing the argument map.

parser_error()

(not exported) (since OTP 26.0)

-type parser_error() ::
          {Path :: cmd_path(),
           Expected :: argument() | undefined,
           Actual :: string() | undefined,
           Details :: unicode:chardata()}.

Returned from parse/2,3 when the user input cannot be parsed according to the command specification.

First element is the path to the command that was considered when the parser detected an error. Second element, Expected, is the argument specification that caused an error. It could be undefined, meaning that Actual argument had no corresponding specification in the arguments list for the current command.

When Actual is set to undefined, it means that a required argument is missing from the command line. If both Expected and Actual have values, it means validation error.

Use format_error/1 to generate a human-readable error description, unless there is a need to provide localised error messages.

parser_options()

(not exported) (since OTP 26.0)

-type parser_options() ::
          #{prefixes => [char()],
            default => term(),
            progname => string() | atom(),
            command => cmd_path(),
            columns => pos_integer()}.

Options changing parser behaviour.

prefixes - Changes the option prefix (the default is -).

default - Specifies the default value for all optional arguments. When this field is set, resulting argument map will contain all argument names. Useful for easy pattern matching on the argument map in the handler function.

progname - Specifies the program (root command) name. Returned as the first element of the command path, and printed in help/usage text. It is recommended to have this value set, otherwise the default one is determined with init:get_argument(progname) and is often set to erl instead of the actual script name.

command - Specifies the path to the nested command for help/2. Useful to limit output for complex utilities with multiple commands, and used by the default error handling logic.

columns - Specifies the help/usage text width (characters) for help/2. Default value is 80.

Functions

format_error(Reason)

(since OTP 26.0)

-spec format_error(Reason :: parser_error()) -> unicode:chardata().

Generates human-readable text for parser error. Does not include help/usage information, and does not provide localisation.

help(Command)

(since OTP 26.0)

-spec help(command()) -> string().

Equivalent to help/2.

help(Command, Options)

(since OTP 26.0)

-spec help(command(), parser_options()) -> unicode:chardata().

Generates help/usage information text for the command supplied, or any nested command when command option is specified. Arguments are displayed in the same order as specified in Command. Does not provide localisation. Expects progname to be set, otherwise defaults to return value of init:get_argument(progname).

parse(Args, Command)

(since OTP 26.0)

-spec parse(args(), command()) -> parse_result().

Equivalent to parse/3.

parse(Args, Command, Options)

(since OTP 26.0)

-spec parse(args(), command(), Options :: parser_options()) -> parse_result().

Parses command line arguments according to the command specification. Raises an exception if the command specification is not valid. Use erl_error:format_exception/3,4 to see a friendlier message. Invalid command line input does not raise an exception, but makes parse/2,3 to return a tuple {error, parser_error()}.

This function does not call command handler.

run(Args, Command, Options)

(since OTP 26.0)

-spec run(args(), command(), parser_options()) -> term().

Parses command line arguments and calls the matching command handler. Prints human-readable error, help/usage information for the discovered command, and halts the emulator with code 1 if there is any error in the command specification or user-provided command line input.

Warning

This function is designed to work as an entry point to a standalone escript. Therefore, it halts the emulator for any error detected. Do not use this function through remote procedure call, or it may result in an unexpected shutdown of a remote node.

escript

This module provides functions to create and inspect escripts.

See the escript program documentation for more details on how to use escripts.

Summary

Types

comment()

emu_args()

Any arguments that should be passed to erl when starting.

extract_option()

section()

section_name()

shebang()

The initial #! line.

zip_file()

Functions

create(File, Options)

Creates an escript from a list of sections.

extract(File, Options)

Parses an escript and extracts its sections. This is the reverse of create/2.

script_name()

Returns the name of the escript that is executed.

Types

comment()

(not exported)

-type comment() :: string().

emu_args()

(not exported)

-type emu_args() :: string().

Any arguments that should be passed to erl when starting.

extract_option()

(not exported)

-type extract_option() :: compile_source | {section, [section_name()]}.

section()

(not exported)

-type section() ::
          shebang |
          {shebang, shebang() | default | undefined} |
          comment |
          {comment, comment() | default | undefined} |
          {emu_args, emu_args() | undefined} |
          {source, file:filename() | binary()} |
          {beam, file:filename() | binary()} |
          {archive, zip:filename() | binary()} |
          {archive, [zip_file()], [zip:create_option()]}.

section_name()

(not exported)

-type section_name() :: shebang | comment | emu_args | body.

shebang()

(not exported)

-type shebang() :: string().

The initial #! line.

For example:

#!/usr/bin/env escript

zip_file()

(not exported)

-type zip_file() ::
          zip:filename() | {zip:filename(), binary()} | {zip:filename(), binary(), file:file_info()}.

Functions

create(File, Options)

-spec create(file:filename() | binary(), [section()]) -> ok | {ok, binary()} | {error, term()}.

Creates an escript from a list of sections.

The sections can be specified in any order. An escript begins with an optional Header followed by a mandatory Body. If the header is present, it does always begin with a shebang, possibly followed by a comment and emu_args. The shebang defaults to "/usr/bin/env escript". The comment defaults to "This is an -*- erlang -*- file". The created escript can either be returned as a binary or written to file.

As an example of how the function can be used, we create an interpreted escript that uses emu_args to set some emulator flag. In this case, it happens to set number of schedulers with +S3. We also extract the different sections from the newly created script:

> Source = "%% Demo\nmain(_Args) ->\n    io:format(\"~p\",[erlang:system_info(schedulers)]).\n".
"%% Demo\nmain(_Args) ->\n    io:format(erlang:system_info(schedulers)).\n"
> io:format("~s\n", [Source]).
%% Demo
main(_Args) ->
    io:format(erlang:system_info(schedulers)).

ok
> {ok, Bin} = escript:create(binary, [shebang, comment, {emu_args, "+S3"},
                                      {source, list_to_binary(Source)}]).
{ok,<<"#!/usr/bin/env escript\n%% This is an -*- erlang -*- file\n%%!+S3"...>>}
> file:write_file("demo.escript", Bin).
ok
> os:cmd("escript demo.escript").
"3"
> escript:extract("demo.escript", []).
{ok,[{shebang,default}, {comment,default}, {emu_args,"+S3"},
     {source,<<"%% Demo\nmain(_Args) ->\n    io:format(erlang:system_info(schedu"...>>}]}

An escript without header can be created as follows:

> file:write_file("demo.erl",
                  ["%% demo.erl\n-module(demo).\n-export([main/1]).\n\n", Source]).
ok
> {ok, _, BeamCode} = compile:file("demo.erl", [binary, debug_info]).
{ok,demo,
    <<70,79,82,49,0,0,2,208,66,69,65,77,65,116,111,109,0,0,0,
      79,0,0,0,9,4,100,...>>}
> escript:create("demo.beam", [{beam, BeamCode}]).
ok
> escript:extract("demo.beam", []).
{ok,[{shebang,undefined}, {comment,undefined}, {emu_args,undefined},
     {beam,<<70,79,82,49,0,0,3,68,66,69,65,77,65,116,
             111,109,0,0,0,83,0,0,0,9,...>>}]}
> os:cmd("escript demo.beam").
"true"

Here we create an archive script containing both Erlang code and Beam code, then we iterate over all files in the archive and collect their contents and some information about them:

> {ok, SourceCode} = file:read_file("demo.erl").
{ok,<<"%% demo.erl\n-module(demo).\n-export([main/1]).\n\n%% Demo\nmain(_Arg"...>>}
> escript:create("demo.escript",
                 [shebang,
                  {archive, [{"demo.erl", SourceCode},
                             {"demo.beam", BeamCode}], []}]).
ok
> {ok, [{shebang,default}, {comment,undefined}, {emu_args,undefined},
     {archive, ArchiveBin}]} = escript:extract("demo.escript", []).
{ok,[{shebang,default}, {comment,undefined}, {emu_args,undefined},
     {{archive,<<80,75,3,4,20,0,0,0,8,0,118,7,98,60,105,
                152,61,93,107,0,0,0,118,0,...>>}]}
> file:write_file("demo.zip", ArchiveBin).
ok
> zip:foldl(fun(N, I, B, A) -> [{N, I(), B()} | A] end, [], "demo.zip").
{ok,[{"demo.beam",
      {file_info,748,regular,read_write,
                 {{2010,3,2},{0,59,22}},
                 {{2010,3,2},{0,59,22}},
                 {{2010,3,2},{0,59,22}},
                 54,1,0,0,0,0,0},
      <<70,79,82,49,0,0,2,228,66,69,65,77,65,116,111,109,0,0,0,
        83,0,0,...>>},
     {"demo.erl",
      {file_info,118,regular,read_write,
                 {{2010,3,2},{0,59,22}},
                 {{2010,3,2},{0,59,22}},
                 {{2010,3,2},{0,59,22}},
                 54,1,0,0,0,0,0},
      <<"%% demo.erl\n-module(demo).\n-export([main/1]).\n\n%% Demo\nmain(_Arg"...>>}]}

extract(File, Options)

-spec extract(file:filename(), [extract_option()]) -> {ok, [section()]} | {error, term()}.

Parses an escript and extracts its sections. This is the reverse of create/2.

All sections are returned even if they do not exist in the escript. If a particular section happens to have the same value as the default value, the extracted value is set to the atom default. If a section is missing, the extracted value is set to the atom undefined.

Option compile_source only affects the result if the escript contains source code. In this case the Erlang code is automatically compiled and {source, BeamCode} is returned instead of {source, SourceCode}.

Example:

> escript:create("demo.escript",
                 [shebang, {archive, [{"demo.erl", SourceCode},
                                      {"demo.beam", BeamCode}], []}]).
ok
> {ok, [{shebang,default}, {comment,undefined}, {emu_args,undefined},
     {archive, ArchiveBin}]} =
              escript:extract("demo.escript", []).
{ok,[{{archive,<<80,75,3,4,20,0,0,0,8,0,118,7,98,60,105,
                152,61,93,107,0,0,0,118,0,...>>}
     {emu_args,undefined}]}

script_name()

-spec script_name() -> string().

Returns the name of the escript that is executed.

If the function is invoked outside the context of an escript, the behavior is undefined.

peer

Start and control linked Erlang nodes.

This module provides functions for starting linked Erlang nodes. The node spawning new nodes is called origin, and newly started nodes are peer nodes, or peers. A peer node automatically terminates when it loses the control connection to the origin. This connection could be an Erlang distribution connection, or an alternative - TCP or standard I/O. The alternative connection provides a way to execute remote procedure calls even when Erlang Distribution is not available, allowing to test the distribution itself.

Peer node terminal input/output is relayed through the origin. If a standard I/O alternative connection is requested, console output also goes via the origin, allowing debugging of node startup and boot script execution (see -init_debug). File I/O is not redirected, contrary to slave behaviour.

The peer node can start on the same or a different host (via ssh) or in a separate container (for example Docker). When the peer starts on the same host as the origin, it inherits the current directory and environment variables from the origin.

Note

This module is designed to facilitate multi-node testing with Common Test. Use the ?CT_PEER() macro to start a linked peer node according to Common Test conventions: crash dumps written to specific location, node name prefixed with module name, calling function, and origin OS process ID). Use random_name/1 to create sufficiently unique node names if you need more control.

A peer node started without alternative connection behaves similarly to slave. When an alternative connection is requested, the behaviour is similar to test_server:start_node(Name, peer, Args).

Example

The following example implements a test suite starting extra Erlang nodes. It employs a number of techniques to speed up testing and reliably shut down peer nodes:

peers start linked to test runner process. If the test case fails, the peer node is stopped automatically, leaving no rogue nodes running in the background

arguments used to start the peer are saved in the control process state for manual analysis. If the test case fails, the CRASH REPORT contains these arguments

multiple test cases can run concurrently speeding up overall testing process, peer node names are unique even when there are multiple instances of the same test suite running in parallel

-module(my_SUITE).
-behaviour(ct_suite).
-export([all/0, groups/0]).
-export([basic/1, args/1, named/1, restart_node/1, multi_node/1]).

-include_lib("common_test/include/ct.hrl").

groups() ->
    [{quick, [parallel],
        [basic, args, named, restart_node, multi_node]}].

all() ->
    [{group, quick}].

basic(Config) when is_list(Config) ->
    {ok, Peer, _Node} = ?CT_PEER(),
    peer:stop(Peer).

args(Config) when is_list(Config) ->
    %% specify additional arguments to the new node
    {ok, Peer, _Node} = ?CT_PEER(["-emu_flavor", "smp"]),
    peer:stop(Peer).

named(Config) when is_list(Config) ->
    %% pass test case name down to function starting nodes
    Peer = start_node_impl(named_test),
    peer:stop(Peer).

start_node_impl(ActualTestCase) ->
    {ok, Peer, Node} = ?CT_PEER(#{name => ?CT_PEER_NAME(ActualTestCase)}),
    %% extra setup needed for multiple test cases
    ok = rpc:call(Node, application, set_env, [kernel, key, value]),
    Peer.

restart_node(Config) when is_list(Config) ->
    Name = ?CT_PEER_NAME(),
    {ok, Peer, Node} = ?CT_PEER(#{name => Name}),
    peer:stop(Peer),
    %% restart the node with the same name as before
    {ok, Peer2, Node} = ?CT_PEER(#{name => Name, args => ["+fnl"]}),
    peer:stop(Peer2).

The next example demonstrates how to start multiple nodes concurrently:

multi_node(Config) when is_list(Config) ->
    Peers = [?CT_PEER(#{wait_boot => {self(), tag}})
        || _ <- lists:seq(1, 4)],
    %% wait for all nodes to complete boot process, get their names:
    _Nodes = [receive {tag, {started, Node, Peer}} -> Node end
        || {ok, Peer} <- Peers],
    [peer:stop(Peer) || {ok, Peer} <- Peers].

Start a peer on a different host. Requires ssh key-based authentication set up, allowing "another_host" connection without password prompt.

Ssh = os:find_executable("ssh"),
peer:start_link(#{exec => {Ssh, ["another_host", "erl"]},
    connection => standard_io}),

The following Common Test case demonstrates Docker integration, starting two containers with hostnames "one" and "two". In this example Erlang nodes running inside containers form an Erlang cluster.

docker(Config) when is_list(Config) ->
    Docker = os:find_executable("docker"),
    PrivDir = proplists:get_value(priv_dir, Config),
    build_release(PrivDir),
    build_image(PrivDir),

    %% start two Docker containers
    {ok, Peer, Node} = peer:start_link(#{name => lambda,
        connection => standard_io,
        exec => {Docker, ["run", "-h", "one", "-i", "lambda"]}}),
    {ok, Peer2, Node2} = peer:start_link(#{name => lambda,
        connection => standard_io,
        exec => {Docker, ["run", "-h", "two", "-i", "lambda"]}}),

    %% find IP address of the second node using alternative connection RPC
    {ok, Ips} = peer:call(Peer2, inet, getifaddrs, []),
    {"eth0", Eth0} = lists:keyfind("eth0", 1, Ips),
    {addr, Ip} = lists:keyfind(addr, 1, Eth0),

    %% make first node to discover second one
    ok = peer:call(Peer, inet_db, set_lookup, [[file]]),
    ok = peer:call(Peer, inet_db, add_host, [Ip, ["two"]]),

    %% join a cluster
    true = peer:call(Peer, net_kernel, connect_node, [Node2]),
    %% verify that second peer node has only the first node visible
    [Node] = peer:call(Peer2, erlang, nodes, []),

    %% stop peers, causing containers to also stop
    peer:stop(Peer2),
    peer:stop(Peer).

build_release(Dir) ->
    %% load sasl.app file, otherwise application:get_key will fail
    application:load(sasl),
    %% create *.rel - release file
    RelFile = filename:join(Dir, "lambda.rel"),
    Release = {release, {"lambda", "1.0.0"},
        {erts, erlang:system_info(version)},
        [{App, begin {ok, Vsn} = application:get_key(App, vsn), Vsn end}
            || App <- [kernel, stdlib, sasl]]},
    ok = file:write_file(RelFile, list_to_binary(lists:flatten(
        io_lib:format("~tp.", [Release])))),
    RelFileNoExt = filename:join(Dir, "lambda"),

    %% create boot script
    {ok, systools_make, []} = systools:make_script(RelFileNoExt,
        [silent, {outdir, Dir}]),
    %% package release into *.tar.gz
    ok = systools:make_tar(RelFileNoExt, [{erts, code:root_dir()}]).

build_image(Dir) ->
    %% Create Dockerfile example, working only for Ubuntu 20.04
    %% Expose port 4445, and make Erlang distribution to listen
    %%  on this port, and connect to it without EPMD
    %% Set cookie on both nodes to be the same.
    BuildScript = filename:join(Dir, "Dockerfile"),
    Dockerfile =
      "FROM ubuntu:20.04 as runner\n"
      "EXPOSE 4445\n"
      "WORKDIR /opt/lambda\n"
      "COPY lambda.tar.gz /tmp\n"
      "RUN tar -zxvf /tmp/lambda.tar.gz -C /opt/lambda\n"
      "ENTRYPOINT [\"/opt/lambda/erts-" ++ erlang:system_info(version) ++
      "/bin/dyn_erl\", \"-boot\", \"/opt/lambda/releases/1.0.0/start\","
      " \"-kernel\", \"inet_dist_listen_min\", \"4445\","
      " \"-erl_epmd_port\", \"4445\","
      " \"-setcookie\", \"secret\"]\n",
    ok = file:write_file(BuildScript, Dockerfile),
    os:cmd("docker build -t lambda " ++ Dir).

Summary

Types

connection()

Alternative connection between the origin and the peer. When the connection closes, the peer node terminates automatically.

disconnect_timeout()

Disconnect timeout. See stop().

exec()

Overrides executable to start peer nodes with.

peer_state()

Peer node state.

server_ref()

Identifies the controlling process of a peer node.

start_options()

Options that can be used when starting a peer node through start/1 and start_link/0,1.

wait_boot()

Specifies start/start_link timeout in milliseconds. Can be set to false, allowing the peer to start asynchronously. If {Pid, Tag} is specified instead of a timeout, the peer will send Tag to the requested process.

Functions

call(Dest, Module, Function, Args)

Equivalent to call(Dest, Module, Function, Args, 5000).

call(Dest, Module, Function, Args, Timeout)

Uses the alternative connection to evaluate apply(Module, Function, Args) on the peer node and returns the corresponding value Result.

cast(Dest, Module, Function, Args)

Uses the alternative connection to evaluate apply(Module, Function, Args) on the peer node. No response is delivered to the calling process.

get_state(Dest)

Returns the peer node state.

random_name()

Equivalent to random_name(peer).

random_name(Prefix)

Creates a sufficiently unique node name for the current host, combining a prefix, a unique number, and the current OS process ID.

send(Dest, To, Message)

Uses the alternative connection to send Message to a process on the the peer node.

start(Options)

Starts a peer node with the specified start_options/0. Returns the controlling process and the full peer node name, unless wait_boot is not requested and the host name is not known in advance.

start_link()

The same as start_link(#{name => random_name()}).

start_link(Options)

Starts a peer node in the same way as start/1, except that the peer node is linked to the currently executing process. If that process terminates, the peer node also terminates.

stop(Dest)

Stops a peer node. How the node is stopped depends on the shutdown option passed when starting the peer node. Currently the following shutdown options are supported

Types

connection()

(not exported) (since OTP 25.0)

-type connection() :: Port :: 0..65535 | {inet:ip_address(), 0..65535} | standard_io.

Alternative connection between the origin and the peer. When the connection closes, the peer node terminates automatically.

If the peer_down startup flag is set to crash, the controlling process on the origin node exits with corresponding reason, effectively providing a two-way link.

When connection is set to a port number, the origin starts listening on the requested TCP port, and the peer node connects to the port. When it is set to an {IP, Port} tuple, the origin listens only on the specified IP. The port number can be set to 0 for automatic selection.

Using the standard_io alternative connection starts the peer attached to the origin (other connections use -detached flag to erl). In this mode peer and origin communicate via stdin/stdout.

disconnect_timeout()

(since OTP 25.0)

-type disconnect_timeout() :: 1000..4294967295 | infinity.

Disconnect timeout. See stop().

exec()

(since OTP 25.0)

-type exec() :: file:name() | {file:name(), [string()]}.

Overrides executable to start peer nodes with.

By default it is the path to "erl", taken from init:get_argument(progname). If progname is not known, peer makes best guess given the current ERTS version.

When a tuple is passed, the first element is the path to executable, and the second element is prepended to the final command line. This can be used to start peers on a remote host or in a Docker container. See the examples above.

This option is useful for testing backwards compatibility with previous releases, installed at specific paths, or when the Erlang installation location is missing from the PATH.

peer_state()

(since OTP 25.0)

-type peer_state() :: booting | running | {down, Reason :: term()}.

Peer node state.

server_ref()

(since OTP 25.0)

-type server_ref() :: pid().

Identifies the controlling process of a peer node.

start_options()

(since OTP 25.0)

-type start_options() ::
          #{name => atom() | string(),
            longnames => boolean(),
            host => string(),
            peer_down => stop | continue | crash,
            connection => connection(),
            exec => exec(),
            detached => boolean(),
            args => [string()],
            post_process_args => fun(([string()]) -> [string()]),
            env => [{string(), string()}],
            wait_boot => wait_boot(),
            shutdown => close | halt | {halt, disconnect_timeout()} | disconnect_timeout()}.

Options that can be used when starting a peer node through start/1 and start_link/0,1.

name - Node name (the part before "@"). When name is not specified, but host is, peer follows compatibility behaviour and uses the origin node name.

longnames - Use long names to start a node. Default is taken from the origin using net_kernel:longnames(). If the origin is not distributed, short names is the default.

host - Enforces a specific host name. Can be used to override the default behaviour and start "node@localhost" instead of "node@realhostname".

peer_down - Defines the peer control process behaviour when the control connection is closed from the peer node side (for example when the peer crashes or dumps core). When set to stop (default), a lost control connection causes the control process to exit normally. Setting peer_down to continue keeps the control process running, and crash will cause the controlling process to exit abnormally.

connection - Alternative connection specification. See the connection datatype.

exec - Alternative mechanism to start peer nodes with, for example, ssh instead of the default bash.

detached - Defines whether to pass the -detached flag to the started peer. This option cannot be set to false using the standard_io alternative connection type. Default is true.

args - Extra command line arguments to append to the "erl" command. Arguments are passed as is, no escaping or quoting is needed or accepted.

post_process_args - Allows the user to change the arguments passed to exec before the peer is started. This can for example be useful when the exec program wants the arguments to "erl" as a single argument. Example:

peer:start(#{ name => peer:random_name(),
  exec => {os:find_executable("bash"),["-c","erl"]},
  post_process_args =>
     fun(["-c"|Args]) -> ["-c", lists:flatten(lists:join($\s, Args))] end
  }).

env - List of environment variables with their values. This list is applied to a locally started executable. If you need to change the environment of the remote peer, adjust args to contain -env ENV_KEY ENV_VALUE.

wait_boot - Specifies the start/start_link timeout. See wait_boot datatype.

shutdown - Specifies the peer node stopping behaviour. See stop().

wait_boot()

(not exported) (since OTP 25.0)

-type wait_boot() :: timeout() | {pid(), Tag :: term()} | false.

Specifies start/start_link timeout in milliseconds. Can be set to false, allowing the peer to start asynchronously. If {Pid, Tag} is specified instead of a timeout, the peer will send Tag to the requested process.

The default is 15_000 ms.

Functions

call(Dest, Module, Function, Args)

(since OTP 25.0)

-spec call(Dest :: server_ref(), Module :: module(), Function :: atom(), Args :: [term()]) ->
              Result :: term().

Equivalent to call(Dest, Module, Function, Args, 5000).

call(Dest, Module, Function, Args, Timeout)

(since OTP 25.0)

-spec call(Dest :: server_ref(),
           Module :: module(),
           Function :: atom(),
           Args :: [term()],
           Timeout :: timeout()) ->
              Result :: term().

Uses the alternative connection to evaluate apply(Module, Function, Args) on the peer node and returns the corresponding value Result.

Timeout is an integer representing the timeout in milliseconds or the atom infinity which prevents the operation from ever timing out.

When an alternative connection is not requested, this function will raise exit signal with the noconnection reason. Use erpc module to communicate over Erlang distribution.

cast(Dest, Module, Function, Args)

(since OTP 25.0)

-spec cast(Dest :: server_ref(), Module :: module(), Function :: atom(), Args :: [term()]) -> ok.

Uses the alternative connection to evaluate apply(Module, Function, Args) on the peer node. No response is delivered to the calling process.

peer:cast/4 fails silently when the alternative connection is not configured. Use erpc module to communicate over Erlang distribution.

get_state(Dest)

(since OTP 25.0)

-spec get_state(Dest :: server_ref()) -> peer_state().

Returns the peer node state.

The initial state is booting; the node stays in that state until then boot script is complete, and then the node progresses to running. If the node stops (gracefully or not), the state changes to down.

random_name()

(since OTP 25.0)

-spec random_name() -> string().

Equivalent to random_name(peer).

random_name(Prefix)

(since OTP 25.0)

-spec random_name(Prefix :: string() | atom()) -> string().

Creates a sufficiently unique node name for the current host, combining a prefix, a unique number, and the current OS process ID.

Note

Use the ?CT_PEER(["erl_arg1"]) macro provided by Common Test -include_lib("common_test/include/ct.hrl") for convenience. It starts a new peer using Erlang distribution as the control channel, supplies thes calling module's code path to the peer, and uses the calling function name for the name prefix.

send(Dest, To, Message)

(since OTP 25.0)

-spec send(Dest :: server_ref(), To :: pid() | atom(), Message :: term()) -> ok.

Uses the alternative connection to send Message to a process on the the peer node.

Silently fails if no alternative connection is configured. The process can be referenced by process ID or registered name.

start(Options)

(since OTP 25.0)

-spec start(start_options()) -> {ok, pid()} | {ok, pid(), node()} | {error, Reason}
               when Reason :: term().

Starts a peer node with the specified start_options/0. Returns the controlling process and the full peer node name, unless wait_boot is not requested and the host name is not known in advance.

start_link()

(since OTP 25.0)

-spec start_link() -> {ok, pid(), node()} | {error, Reason :: term()}.

The same as start_link(#{name => random_name()}).

start_link(Options)

(since OTP 25.0)

-spec start_link(start_options()) -> {ok, pid()} | {ok, pid(), node()} | {error, Reason}
                    when Reason :: term().

Starts a peer node in the same way as start/1, except that the peer node is linked to the currently executing process. If that process terminates, the peer node also terminates.

Accepts start_options/0. Returns the controlling process and the full peer node name, unless wait_boot is not requested and host name is not known in advance.

When the standard_io alternative connection is requested, and wait_boot is not set to false, a failed peer boot sequence causes the caller to exit with the {boot_failed, {exit_status, ExitCode}} reason.

stop(Dest)

(since OTP 25.0)

-spec stop(Dest :: server_ref()) -> ok.

Stops a peer node. How the node is stopped depends on the shutdown option passed when starting the peer node. Currently the following shutdown options are supported:

halt - This is the default shutdown behavior. It behaves as shutdown option {halt, DefaultTimeout} where DefaultTimeout currently equals 5000.

{halt, Timeout :: disconnect_timeout()} - Triggers a call to erlang:halt() on the peer node and then waits for the Erlang distribution connection to the peer node to be taken down. If this connection has not been taken down after Timeout milliseconds, it will forcefully be taken down by peer:stop/1. See the warning below for more info about this.

Timeout :: disconnect_timeout() - Triggers a call to init:stop() on the peer node and then waits for the Erlang distribution connection to the peer node to be taken down. If this connection has not been taken down after Timeout milliseconds, it will forcefully be taken down by peer:stop/1. See the warning below for more info about this.

close - Close the control connection to the peer node and return. This is the fastest way for the caller of peer:stop/1 to stop a peer node.

Note that if the Erlang distribution connection is not used as control connection it might not have been taken down when peer:stop/1 returns. Also note that the warning below applies when the Erlang distribution connection is used as control connection.

Warning

In the cases where the Erlang distribution connection is taken down by peer:stop/1, other code independent of the peer code might react to the connection loss before the peer node is stopped which might cause undesirable effects. For example, global might trigger even more Erlang distribution connections to other nodes to be taken down. The potential undesirable effects are, however, not limited to this. It is hard to say what the effects will be since these effects can be caused by any code with links or monitors to something on the origin node, or code monitoring the connection to the origin node.

slave

This module provides functions for starting Erlang slave nodes.

All slave nodes that are started by a master terminate automatically when the master terminates. All terminal output produced at the slave is sent back to the master node. File I/O is done through the master.

Slave nodes on other hosts than the current one are started with the ssh program. The user must be allowed to ssh to the remote hosts without being prompted for a password. This can be arranged in a number of ways (for details, see the ssh documentation). A slave node started on the same host as the master inherits certain environment values from the master, such as the current directory and the environment variables. For what can be assumed about the environment when a slave is started on another host, see the documentation for the ssh program.

An alternative to the ssh program can be specified on the command line to erl(1) as follows:

-rsh Program

Note that the command specified with the -rsh flag is treated as a file name which may contain spaces. It is thus not possible to include any command line options. The remote node will be launched as "$RSH" "$REMOTE_HOSTNAME" erl -detached -noinput ..., so the erl command must be found in the path on the remote host.

The slave node is to use the same file system at the master. At least, Erlang/OTP is to be installed in the same place on both computers and the same version of Erlang is to be used.

A node running on Windows can only start slave nodes on the host on which it is running.

The master node must be alive.

Summary

Functions

pseudo([Master | ServerList]) deprecated

Calls pseudo(Master, ServerList). If you want to start a node from the command line and set up a number of pseudo servers, an Erlang runtime system can be started as follows

pseudo(Master, ServerList) deprecated

Starts a number of pseudo servers. A pseudo server is a server with a registered name that does nothing but pass on all message to the real server that executes at a master node. A pseudo server is an intermediary that only has the same registered name as the real server.

relay(Pid) deprecated

Runs a pseudo server. This function never returns any value and the process that executes the function receives messages. All messages received are simply passed on to Pid.

start(Host) deprecated

Equivalent to start(Host, Name) where Name is the same as the node that executes this call.

start(Host, Name) deprecated

Equivalent to start(Host, Name, []).

start(Host, Name, Args) deprecated

Starts a slave node on host Host. Host names need not necessarily be specified as fully qualified names; short names can also be used. This is the same condition that applies to names of distributed Erlang nodes.

start_link(Host) deprecated

Equivalent to start_link/3.

start_link(Host, Name) deprecated

Equivalent to start_link/3.

start_link(Host, Name, Args) deprecated

Starts a slave node in the same way as start/1,2,3, except that the slave node is linked to the currently executing process. If that process terminates, the slave node also terminates.

stop(Node) deprecated

Stops (kills) a node.

Functions

pseudo([Master | ServerList])

This function is deprecated. slave:pseudo/1 is deprecated; use the 'peer' module instead.

-spec pseudo([Master :: node() | (ServerList :: [atom()])]) -> ok.

Calls pseudo(Master, ServerList). If you want to start a node from the command line and set up a number of pseudo servers, an Erlang runtime system can be started as follows:

% erl -name abc -s slave pseudo klacke@super x --

pseudo(Master, ServerList)

This function is deprecated. slave:pseudo/2 is deprecated; use the 'peer' module instead.

-spec pseudo(Master, ServerList) -> ok when Master :: node(), ServerList :: [atom()].

Starts a number of pseudo servers. A pseudo server is a server with a registered name that does nothing but pass on all message to the real server that executes at a master node. A pseudo server is an intermediary that only has the same registered name as the real server.

For example, if you have started a slave node N and want to execute pxw graphics code on this node, you can start server pxw_server as a pseudo server at the slave node. This is illustrated as follows:

rpc:call(N, slave, pseudo, [node(), [pxw_server]]).

relay(Pid)

This function is deprecated. slave:relay/1 is deprecated; use the 'peer' module instead.

-spec relay(Pid) -> no_return() when Pid :: pid().

Runs a pseudo server. This function never returns any value and the process that executes the function receives messages. All messages received are simply passed on to Pid.

start(Host)

This function is deprecated. slave:start/1 is deprecated; use the 'peer' module instead.

-spec start(Host) -> {ok, Node} | {error, Reason}
               when
                   Host :: inet:hostname(),
                   Node :: node(),
                   Reason :: timeout | no_rsh | {already_running, Node}.

Equivalent to start(Host, Name) where Name is the same as the node that executes this call.

start(Host, Name)

This function is deprecated. slave:start/2 is deprecated; use the 'peer' module instead.

-spec start(Host, Name) -> {ok, Node} | {error, Reason}
               when
                   Host :: inet:hostname(),
                   Name :: atom() | string(),
                   Node :: node(),
                   Reason :: timeout | no_rsh | {already_running, Node}.

Equivalent to start(Host, Name, []).

start(Host, Name, Args)

This function is deprecated. slave:start/3 is deprecated; use the 'peer' module instead.

-spec start(Host, Name, Args) -> {ok, Node} | {error, Reason}
               when
                   Host :: inet:hostname(),
                   Name :: atom() | string(),
                   Args :: string(),
                   Node :: node(),
                   Reason :: timeout | no_rsh | {already_running, Node}.

Starts a slave node on host Host. Host names need not necessarily be specified as fully qualified names; short names can also be used. This is the same condition that applies to names of distributed Erlang nodes.

The name of the started node becomes Name@Host.

The slave node resets its io:user/0 process so that all terminal I/O that is produced at the slave is automatically relayed to the master. Also, the file server is relayed to the master.

Argument Args is used to set erl command-line arguments. It is passed to the new node and can be used for a variety of purposes; see erl(1).

As an example, suppose that you want to start a slave node at host H with node name Name@H and want the slave node to have the following properties:

Directory Dir is to be added to the code path.

The Mnesia directory is to be set to M.

The Unix DISPLAY environment variable is to be set to the display of the master node.

The following code is executed to achieve this:

E = " -env DISPLAY " ++ net_adm:localhost() ++ ":0 ",
Arg = "-mnesia_dir " ++ M ++ " -pa " ++ Dir ++ E,
slave:start(H, Name, Arg).

The function returns {ok, Node}, where Node is the name of the new node, otherwise {error, Reason}, where Reason can be one of:

timeout - The master node failed to get in contact with the slave node. This can occur in a number of circumstances:

Erlang/OTP is not installed on the remote host.

The file system on the other host has a different structure to the the master.

The Erlang nodes have different cookies.

no_rsh - No remote shell program was found on the computer. Note that ssh is used by default, but this can be overridden with the -rsh flag.

{already_running, Node} - A node with name Name@Host already exists.

start_link(Host)

This function is deprecated. slave:start_link/1 is deprecated; use the 'peer' module instead.

-spec start_link(Host) -> {ok, Node} | {error, Reason}
                    when
                        Host :: inet:hostname(),
                        Node :: node(),
                        Reason :: timeout | no_rsh | {already_running, Node}.

Equivalent to start_link/3.

start_link(Host, Name)

This function is deprecated. slave:start_link/2 is deprecated; use the 'peer' module instead.

-spec start_link(Host, Name) -> {ok, Node} | {error, Reason}
                    when
                        Host :: inet:hostname(),
                        Name :: atom() | string(),
                        Node :: node(),
                        Reason :: timeout | no_rsh | {already_running, Node}.

Equivalent to start_link/3.

start_link(Host, Name, Args)

This function is deprecated. slave:start_link/3 is deprecated; use the 'peer' module instead.

-spec start_link(Host, Name, Args) -> {ok, Node} | {error, Reason}
                    when
                        Host :: inet:hostname(),
                        Name :: atom() | string(),
                        Args :: string(),
                        Node :: node(),
                        Reason :: timeout | no_rsh | {already_running, Node}.

Starts a slave node in the same way as start/1,2,3, except that the slave node is linked to the currently executing process. If that process terminates, the slave node also terminates.

For a description of arguments and return values, see start/1,2,3.

stop(Node)

This function is deprecated. slave:stop/1 is deprecated; use the 'peer' module instead.

-spec stop(Node) -> ok when Node :: node().

Stops (kills) a node.

win32reg

Provides access to the registry on Windows.

This module provides read and write access to the registry on Windows. It is essentially a port driver wrapped around the Win32 API calls for accessing the registry.

The registry is a hierarchical database, used to store various system and software information in Windows. It contains installation data, and is updated by installers and system programs. The Erlang installer updates the registry by adding data that Erlang needs.

The registry contains keys and values. Keys are like the directories in a file system, they form a hierarchy. Values are like files, they have a name and a value, and also a type.

Paths to keys are left to right, with subkeys to the right and backslash between keys. (Remember that backslashes must be doubled in Erlang strings.) Case is preserved but not significant.

For example, "\\hkey_local_machine\\software\\Ericsson\\Erlang\\5.0" is the key for the installation data for the latest Erlang release.

There are six entry points in the Windows registry, top-level keys. They can be abbreviated in this module as follows:

Abbreviation     Registry key
============     ============
hkcr             HKEY_CLASSES_ROOT
current_user     HKEY_CURRENT_USER
hkcu             HKEY_CURRENT_USER
local_machine    HKEY_LOCAL_MACHINE
hklm             HKEY_LOCAL_MACHINE
users            HKEY_USERS
hku              HKEY_USERS
current_config   HKEY_CURRENT_CONFIG
hkcc             HKEY_CURRENT_CONFIG
dyn_data         HKEY_DYN_DATA
hkdd             HKEY_DYN_DATA

The key above can be written as "\\hklm\\software\\ericsson\\erlang\\5.0".

This module uses a current key. It works much like the current directory. From the current key, values can be fetched, subkeys can be listed, and so on.

Under a key, any number of named values can be stored. They have names, types, and data.

win32reg supports storing of the following types:

REG_DWORD, which is an integer

REG_SZ, which is a string

REG_BINARY, which is a binary

Other types can be read, and are returned as binaries.

There is also a "default" value, which has the empty string as name. It is read and written with the atom default instead of the name.

Some registry values are stored as strings with references to environment variables, for example, %SystemRoot%Windows. SystemRoot is an environment variable, and is to be replaced with its value. Function expand/1 is provided so that environment variables surrounded by % can be expanded to their values.

For more information on the Windows registry, see consult the Win32 Programmer's Reference.

See Also

erl_posix_msg, The Windows 95 Registry (book from O'Reilly), Win32 Programmer's Reference (from Microsoft)

Summary

Types

name()

reg_handle()

As returned by open/1.

value()

Functions

change_key(RegHandle, Key)

Changes the current key to another key. Works like cd. The key can be specified as a relative path or as an absolute path, starting with \.

change_key_create(RegHandle, Key)

Creates a key, or just changes to it, if it is already there. Works like a combination of mkdir and cd. Calls the Win32 API function RegCreateKeyEx().

close(RegHandle)

Closes the registry. After that, the RegHandle cannot be used.

current_key(RegHandle)

Returns the path to the current key. This is the equivalent of pwd.

delete_key(RegHandle)

Deletes the current key, if it is valid. Calls the Win32 API function RegDeleteKey(). Notice that this call does not change the current key (unlike change_key_create/2). This means that after the call, the current key is invalid.

delete_value(RegHandle, Name)

Deletes a named value on the current key. The atom default is used for the default value.

expand(String)

Expands a string containing environment variables between percent characters. Anything between two % is taken for an environment variable, and is replaced by the value. Two consecutive % are replaced by one %.

format_error(ErrorId)

Converts a POSIX error code to a string (by calling file:format_error/1).

open(OpenModeList)

Opens the registry for reading or writing. The current key is the root (HKEY_CLASSES_ROOT). Flag read in the mode list can be omitted.

set_value(RegHandle, Name, Value)

Sets the named (or default) value to value. Calls the Win32 API function RegSetValueEx(). The value can be of three types, and the corresponding registry type is used. The supported types are the following

sub_keys(RegHandle)

Returns a list of subkeys to the current key. Calls the Win32 API function EnumRegKeysEx().

value(RegHandle, Name)

Retrieves the named value (or default) on the current key. Registry values of type REG_SZ are returned as strings. Type REG_DWORD values are returned as integers. All other types are returned as binaries.

values(RegHandle)

Retrieves a list of all values on the current key. The values have types corresponding to the registry types, see value/2. Calls the Win32 API function EnumRegValuesEx().

Types

name()

(not exported)

-type name() :: string() | default.

reg_handle()

-opaque reg_handle()

As returned by open/1.

value()

(not exported)

-type value() :: string() | integer() | binary().

Functions

change_key(RegHandle, Key)

-spec change_key(RegHandle, Key) -> ReturnValue
                    when
                        RegHandle :: reg_handle(),
                        Key :: string(),
                        ReturnValue :: ok | {error, ErrorId :: atom()}.

Changes the current key to another key. Works like cd. The key can be specified as a relative path or as an absolute path, starting with \.

change_key_create(RegHandle, Key)

-spec change_key_create(RegHandle, Key) -> ReturnValue
                           when
                               RegHandle :: reg_handle(),
                               Key :: string(),
                               ReturnValue :: ok | {error, ErrorId :: atom()}.

Creates a key, or just changes to it, if it is already there. Works like a combination of mkdir and cd. Calls the Win32 API function RegCreateKeyEx().

The registry must have been opened in write mode.

close(RegHandle)

-spec close(RegHandle) -> ok when RegHandle :: reg_handle().

Closes the registry. After that, the RegHandle cannot be used.

current_key(RegHandle)

-spec current_key(RegHandle) -> ReturnValue
                     when RegHandle :: reg_handle(), ReturnValue :: {ok, string()}.

Returns the path to the current key. This is the equivalent of pwd.

Notice that the current key is stored in the driver, and can be invalid (for example, if the key has been removed).

delete_key(RegHandle)

-spec delete_key(RegHandle) -> ReturnValue
                    when RegHandle :: reg_handle(), ReturnValue :: ok | {error, ErrorId :: atom()}.

Deletes the current key, if it is valid. Calls the Win32 API function RegDeleteKey(). Notice that this call does not change the current key (unlike change_key_create/2). This means that after the call, the current key is invalid.

delete_value(RegHandle, Name)

-spec delete_value(RegHandle, Name) -> ReturnValue
                      when
                          RegHandle :: reg_handle(),
                          Name :: name(),
                          ReturnValue :: ok | {error, ErrorId :: atom()}.

Deletes a named value on the current key. The atom default is used for the default value.

The registry must have been opened in write mode.

expand(String)

-spec expand(String) -> ExpandedString when String :: string(), ExpandedString :: string().

Expands a string containing environment variables between percent characters. Anything between two % is taken for an environment variable, and is replaced by the value. Two consecutive % are replaced by one %.

A variable name that is not in the environment results in an error.

format_error(ErrorId)

-spec format_error(ErrorId) -> ErrorString when ErrorId :: atom(), ErrorString :: string().

Converts a POSIX error code to a string (by calling file:format_error/1).

open(OpenModeList)

-spec open(OpenModeList) -> ReturnValue
              when
                  OpenModeList :: [OpenMode],
                  OpenMode :: read | write,
                  ReturnValue :: {ok, RegHandle} | {error, ErrorId :: enotsup},
                  RegHandle :: reg_handle().

Opens the registry for reading or writing. The current key is the root (HKEY_CLASSES_ROOT). Flag read in the mode list can be omitted.

Use change_key/2 with an absolute path after open.

set_value(RegHandle, Name, Value)

-spec set_value(RegHandle, Name, Value) -> ReturnValue
                   when
                       RegHandle :: reg_handle(),
                       Name :: name(),
                       Value :: value(),
                       ReturnValue :: ok | {error, ErrorId :: atom()}.

Sets the named (or default) value to value. Calls the Win32 API function RegSetValueEx(). The value can be of three types, and the corresponding registry type is used. The supported types are the following:

REG_DWORD for integers

REG_SZ for strings

REG_BINARY for binaries

Other types cannot be added or changed.

The registry must have been opened in write mode.

sub_keys(RegHandle)

-spec sub_keys(RegHandle) -> ReturnValue
                  when
                      RegHandle :: reg_handle(),
                      ReturnValue :: {ok, [SubKey]} | {error, ErrorId :: atom()},
                      SubKey :: string().

Returns a list of subkeys to the current key. Calls the Win32 API function EnumRegKeysEx().

Avoid calling this on the root keys, as it can be slow.

value(RegHandle, Name)

-spec value(RegHandle, Name) -> ReturnValue
               when
                   RegHandle :: reg_handle(),
                   Name :: name(),
                   ReturnValue :: {ok, Value :: value()} | {error, ErrorId :: atom()}.

Retrieves the named value (or default) on the current key. Registry values of type REG_SZ are returned as strings. Type REG_DWORD values are returned as integers. All other types are returned as binaries.

values(RegHandle)

-spec values(RegHandle) -> ReturnValue
                when
                    RegHandle :: reg_handle(),
                    ReturnValue :: {ok, [ValuePair]} | {error, ErrorId :: atom()},
                    ValuePair :: {Name :: name(), Value :: value()}.

Retrieves a list of all values on the current key. The values have types corresponding to the registry types, see value/2. Calls the Win32 API function EnumRegValuesEx().

gen_event behaviour

Generic event handling behavior.

This behavior module provides event handling functionality. It consists of a generic event manager process with any number of event handlers that are added and deleted dynamically.

An event manager implemented using this module has a standard set of interface functions and includes functionality for tracing and error reporting. It also fits into an OTP supervision tree. For more information, see gen_event section in OTP Design Principles.

Each event handler is implemented as a callback module exporting a predefined set of functions. The relationship between the behavior functions and the callback functions is as follows:

gen_event module                   Callback module
----------------                   ---------------
gen_event:start
gen_event:start_monitor
gen_event:start_link       ----->  -

gen_event:add_handler
gen_event:add_sup_handler  ----->  Module:init/1

gen_event:notify
gen_event:sync_notify      ----->  Module:handle_event/2

gen_event:send_request
gen_event:call             ----->  Module:handle_call/2

-                          ----->  Module:handle_info/2

gen_event:delete_handler   ----->  Module:terminate/2

gen_event:swap_handler
gen_event:swap_sup_handler ----->  Module1:terminate/2
                                   Module2:init/1

gen_event:which_handlers   ----->  -

gen_event:stop             ----->  Module:terminate/2

-                          ----->  Module:code_change/3

As each event handler is one callback module, an event manager has many callback modules that are added and deleted dynamically. gen_event is therefore more tolerant of callback module errors than the other behaviors. If a callback function for an installed event handler fails with Reason, or returns a bad value Term, the event manager does not fail. It deletes the event handler by calling callback function Module:terminate/2, giving as argument {error, {'EXIT', Reason}} or {error, Term}, respectively. No other event handler is affected.

A gen_event process handles system messages as described in sys. The sys module can be used for debugging an event manager.

Notice that an event manager does trap exit signals automatically.

The gen_event process can go into hibernation (see erlang:hibernate/3) if a callback function in a handler module specifies hibernate in its return value. This can be useful if the server is expected to be idle for a long time. However, use this feature with care, as hibernation implies at least two garbage collections (when hibernating and shortly after waking up) and is not something you want to do between each event handled by a busy event manager.

Notice that when multiple event handlers are invoked, it is sufficient that one single event handler returns a hibernate request for the whole event manager to go into hibernation.

Unless otherwise stated, all functions in this module fail if the specified event manager does not exist or if bad arguments are specified.

Note

For some important information about distributed signals, see the Blocking Signaling Over Distribution section in the Processes chapter of the Erlang Reference Manual. Blocking signaling can, for example, cause call timeouts in gen_event to be significantly delayed.

See Also

supervisor, sys

Summary

Types

add_handler_ret()

debug_flag()

del_handler_ret()

emgr_name()

Event manager name specification: local, global, or via registered.

emgr_ref()

A reference used to locate an event manager.

format_status()

A map that describes the gen_event process status.

handler()

handler_args()

options()

Options that can be used to configure an event handler when it is started.

request_id()

An opaque request identifier. See send_request/3 for details.

request_id_collection()

An opaque collection of request identifiers (request_id/0).

response_timeout()

Response time-out for an asynchronous call.

start_mon_ret()

start_ret()

Callbacks

code_change(OldVsn, State, Extra)

Update the event handler state after code change.

format_status(Status)

Format/limit the status value.

format_status(Opt, StatusData) deprecated

Format/limit the status value.

handle_call(Request, State)

Handle a call.

handle_event(Event, State)

Handle an event.

handle_info(Info, State)

Handle an info message (regular process message).

init(InitArgs)

Initialize the event handler.

terminate(Args, State)

Handle event handler termination.

Functions

add_handler(EventMgrRef, Handler, Args)

Add a new event handler to an event manager.

add_sup_handler(EventMgrRef, Handler, Args)

Add a new event handler to an event manager, supervised.

call(EventMgrRef, Handler, Request)

Equivalent to call(EventMgrRef, Handler, Request, 5000).

call(EventMgrRef, Handler, Request, Timeout)

Make a synchronous call to an event handler.

check_response(Msg, ReqId)

Check if a received message is a request response.

check_response(Msg, ReqIdCollection, Delete)

Check if a received message is a request response in a collection.

delete_handler(EventMgrRef, Handler, Args)

Deletes an event handler from an event manager.

notify(EventMgrRef, Event)

Send an asynchronous event notification to an event manager.

receive_response(ReqId, Timeout)

Receive a request response.

receive_response(ReqIdCollection, Timeout, Delete)

Receive a request response in a collection.

reqids_add(ReqId, Label, ReqIdCollection)

Store a request identifier in a colletion.

reqids_new()

Create an empty request identifier collection.

reqids_size(ReqIdCollection)

Returns the number of request identifiers in ReqIdCollection.

reqids_to_list(ReqIdCollection)

Convert a request identifier collection to a list.

send_request(EventMgrRef, Handler, Request)

Send an asynchronous call request to an event handler.

send_request(EventMgrRef, Handler, Request, Label, ReqIdCollection)

Send an asynchronous call request to an event handler, storing it in a request identifier collection.

start()

Equivalent to start([]).

start/1

Create a stand-alone event manager process, possibly nameless.

start(EventMgrName, Options)

Create a stand-alone event manager process.

start_link()

Equivalent to start_link([]).

start_link/1

Create an event manager process as part of a supervision tree, possibly nameless.

start_link(EventMgrName, Options)

Create an event manager process as part of a supervision tree.

start_monitor()

Equivalent to start_monitor([]).

start_monitor(EventMgrNameOrOptions)

Creates a stand-alone event manager process, monitored, possibly nameless.

start_monitor(EventMgtName, Options)

Creates a stand-alone event manager process, monitored.

stop(EventMgrRef)

Equivalent to stop(EventMgrRef, normal, infinity).

stop(EventMgrRef, Reason, Timeout)

Stop an event manager.

swap_handler(EventMgrRef, OldHandler, NewHandler)

Replace an event handler.

swap_sup_handler(EventMgrRef, OldHandler, NewHandler)

Replace an event handler, and supervise it.

sync_notify(EventMgrRef, Event)

Send a synchronous event notification to an event manager.

wait_response(ReqId, WaitTime)

Wait for a request resonse.

wait_response(ReqIdCollection, WaitTime, Delete)

Wait for any request response in a collection.

which_handlers(EventMgrRef)

Return all event handlers in an event manager.

Types

add_handler_ret()

-type add_handler_ret() :: ok | term() | {'EXIT', term()}.

debug_flag()

(not exported)

-type debug_flag() :: trace | log | statistics | debug | {logfile, string()}.

del_handler_ret()

-type del_handler_ret() :: ok | term() | {'EXIT', term()}.

emgr_name()

(not exported)

-type emgr_name() :: {local, atom()} | {global, term()} | {via, atom(), term()}.

Event manager name specification: local, global, or via registered.

{local, Name} - the event manager is registered locally as Name using register/2.

{global, GlobalName} - The event manager is registered globally as GlobalName using global:register_name/2. If no name is provided, the event manager is not registered.

{via, Module, ViaName}, the event manager registers with the registry represented by Module. The Module callback is to export the functions register_name/2, unregister_name/1, whereis_name/1, and send/2, which are to behave as the corresponding functions in global. Thus, {via, global, GlobalName} is a valid reference.

emgr_ref()

(not exported)

-type emgr_ref() :: atom() | {atom(), node()} | {global, term()} | {via, atom(), term()} | pid().

A reference used to locate an event manager.

The reference can be any of the following:

The pid of the event manager

Name, if the event manager is locally registered

{Name, Node}, if the event manager is locally registered at another node

{global, GlobalName}, if the event manager is globally registered

{via, Module, ViaName}, if the event manager is registered through an alternative process registry

format_status()

-type format_status() ::
          #{state => term(), message => term(), reason => term(), log => [sys:system_event()]}.

A map that describes the gen_event process status.

The keys are:

state - The internal state of the event handler.

message - The message that caused the event handler to terminate.

reason - The reason that caused the event handler to terminate.

log - The sys log of the server.

New associations may be added into the status map without prior notice.

handler()

-type handler() :: atom() | {atom(), term()}.

handler_args()

-type handler_args() :: term().

options()

(not exported)

-type options() ::
          [{timeout, timeout()} |
           {debug, [debug_flag()]} |
           {spawn_opt, [proc_lib:start_spawn_option()]} |
           {hibernate_after, timeout()}].

Options that can be used to configure an event handler when it is started.

request_id()

-opaque request_id()

An opaque request identifier. See send_request/3 for details.

request_id_collection()

-opaque request_id_collection()

An opaque collection of request identifiers (request_id/0).

Each request identifier can be associated with a label chosen by the user. For more information see reqids_new/0.

response_timeout()

(not exported)

-type response_timeout() :: timeout() | {abs, integer()}.

Response time-out for an asynchronous call.

Used to set a time limit on how long to wait for a response using either receive_response/2, receive_response/3, wait_response/2, or wait_response/3. The time unit used is millisecond. Currently valid values:

0..4294967295 - Timeout relative to current time in milliseconds.

infinity - Infinite timeout. That is, the operation will never time out.

{abs, Timeout} - An absolute Erlang monotonic time timeout in milliseconds. That is, the operation will time out when erlang:monotonic_time(millisecond) returns a value larger than or equal to Timeout. Timeout is not allowed to identify a time further into the future than 4294967295 milliseconds. Identifying the timeout using an absolute timeout value is especially handy when you have a deadline for responses corresponding to a complete collection of requests (request_id_collection/0) , since you do not have to recalculate the relative time until the deadline over and over again.

start_mon_ret()

(not exported)

-type start_mon_ret() :: {ok, {pid(), reference()}} | {error, term()}.

start_ret()

(not exported)

-type start_ret() :: {ok, pid()} | {error, term()}.

Callbacks

code_change(OldVsn, State, Extra)

(optional)

-callback code_change(OldVsn :: term() | {down, term()}, State :: term(), Extra :: term()) ->
                         {ok, NewState :: term()}.

Update the event handler state after code change.

This function is called for an installed event handler that is to update its internal state during a release upgrade/downgrade, that is, when the instruction {update, Module, Change,...}, is specified in the appup file.

For more information, see OTP Design Principles.

For an upgrade, OldVsn is Vsn, and for a downgrade, OldVsn is {down, Vsn}. Vsn is defined by the vsn attribute(s) of the old version of the callback module Module. If no such attribute is defined, the version is the checksum of the Beam file.

State is the internal state of the event handler.

Extra is passed "as is" from the {advanced, Extra} part of the update instruction.

The function is to return the updated internal state.

Note

If a release upgrade/downgrade with Change={advanced, Extra} specified in the .appup file is made when code_change/3 is not implemented the event handler will crash with an undef error reason.

format_status(Status)

(since OTP 25.0) (optional)

-callback format_status(Status) -> NewStatus when Status :: format_status(), NewStatus :: format_status().

Format/limit the status value.

This function is called by a gen_event process in in order to format/limit the server state for debugging and logging purposes.

It is called in the following situations:

One of sys:get_status/1,2 is invoked to get the gen_event status.

The event handler terminates abnormally and gen_event logs an error.

This callback is used to limit the status of the event handler returned by sys:get_status/1,2 or sent to logger.

The callback gets a map Status describing the current status and shall return a map NewStatus with the same keys, but it may transform some values.

Two possible use cases for this callback is to remove sensitive information from the state to prevent it from being printed in log files, or to compact large irrelevant status items that would only clutter the logs.

Example:

format_status(Status) ->
  maps:map(
    fun(state,State) ->
            maps:remove(private_key, State);
       (message,{password, _Pass}) ->
            {password, removed};
       (_,Value) ->
            Value
    end, Status).

Note

This callback is optional, so event handler modules need not export it. If a handler does not export this function, the gen_event module uses the handler state directly for the purposes described below.

If this callback is exported but fails, to hide possibly sensitive data, the default function will instead return the fact that format_status/1 has crashed.

format_status(Opt, StatusData)

(since OTP R14B) (optional)

This callback is deprecated. the callback gen_event:format_status(_,_) is deprecated; use format_status/1 instead.

-callback format_status(Opt, StatusData) -> Status
                           when
                               Opt :: normal | terminate,
                               StatusData :: [PDict | State],
                               PDict :: [{Key :: term(), Value :: term()}],
                               State :: term(),
                               Status :: term().

Format/limit the status value.

This function is called by a gen_event process in in order to format/limit the server state for debugging and logging purposes.

It is called in the following situations:

One of sys:get_status/1,2 is invoked to get the gen_event status. Opt is set to the atom normal for this case.

The event handler terminates abnormally and gen_event logs an error. Opt is set to the atom terminate for this case.

This function is useful for changing the form and appearance of the event handler state for these cases. An event handler callback module wishing to change the sys:get_status/1,2 return value as well as how its state appears in termination error logs, exports an instance of format_status/2 that returns a term describing the current state of the event handler.

PDict is the current value of the process dictionary of gen_event.

State is the internal state of the event handler.

The function is to return Status, a term that change the details of the current state of the event handler. Any term is allowed for Status. The gen_event module uses Status as follows:

When sys:get_status/1,2 is called, gen_event ensures that its return value contains Status in place of the state term of the event handler.

When an event handler terminates abnormally, gen_event logs Status in place of the state term of the event handler.

One use for this function is to return compact alternative state representations to avoid that large state terms are printed in log files.

Note

This callback is optional, so event handler modules need not export it. If a handler does not export this function, the gen_event module uses the handler state directly for the purposes described below.

handle_call(Request, State)

-callback handle_call(Request :: term(), State :: term()) ->
                         {ok, Reply :: term(), NewState :: term()} |
                         {ok, Reply :: term(), NewState :: term(), hibernate} |
                         {swap_handler,
                          Reply :: term(),
                          Args1 :: term(),
                          NewState :: term(),
                          Handler2 :: atom() | {atom(), Id :: term()},
                          Args2 :: term()} |
                         {remove_handler, Reply :: term()}.

Handle a call.

Whenever an event manager receives a request sent using call/3,4, this function is called for the specified event handler to handle the request.

Request is the Request argument of call/3,4.

State is the internal state of the event handler.

The return values are the same as for Module:handle_event/2 except that they also contain a term Reply, which is the reply to the client as the return value of call/3,4.

handle_event(Event, State)

-callback handle_event(Event :: term(), State :: term()) ->
                          {ok, NewState :: term()} |
                          {ok, NewState :: term(), hibernate} |
                          {swap_handler,
                           Args1 :: term(),
                           NewState :: term(),
                           Handler2 :: atom() | {atom(), Id :: term()},
                           Args2 :: term()} |
                          remove_handler.

Handle an event.

Whenever an event manager receives an event sent using notify/2 or sync_notify/2, this function is called for each installed event handler to handle the event.

Event is the Event argument of notify/2 / sync_notify/2.

State is the internal state of the event handler.

If {ok, NewState} or {ok, NewState, hibernate} is returned, the event handler remains in the event manager with the possibly updated internal state NewState.

If {ok, NewState, hibernate} is returned, the event manager also goes into hibernation (by calling proc_lib:hibernate/3), waiting for the next event to occur. It is sufficient that one of the event handlers return {ok, NewState, hibernate} for the whole event manager process to hibernate.

If {swap_handler, Args1, NewState, Handler2, Args2} is returned, the event handler is replaced by Handler2 by first calling Module:terminate(Args1, NewState) and then Module2:init({Args2, Term}), where Term is the return value of Module:terminate/2. For more information, see swap_handler/3.

If remove_handler is returned, the event handler is deleted by calling Module:terminate(remove_handler, State).

handle_info(Info, State)

(optional)

-callback handle_info(Info :: term(), State :: term()) ->
                         {ok, NewState :: term()} |
                         {ok, NewState :: term(), hibernate} |
                         {swap_handler,
                          Args1 :: term(),
                          NewState :: term(),
                          Handler2 :: atom() | {atom(), Id :: term()},
                          Args2 :: term()} |
                         remove_handler.

Handle an info message (regular process message).

This function is called for each installed event handler when an event manager receives any other message than an event or a synchronous request (or a system message).

Info is the received message.

In particular, this callback will be made when a process terminated after calling add_sup_handler/3. Any event handler attached to an event manager which in turn has a supervised handler should expect callbacks of the shape Module:handle_info({'EXIT', Pid, Reason}, State).

For a description of State and possible return values, see Module:handle_event/2.

Note

This callback is optional, so callback modules need not export it. The gen_event module provides a default implementation of this function that logs about the unexpected Info message, drops it and returns {ok, State}.

init(InitArgs)

-callback init(InitArgs :: term()) ->
                  {ok, State :: term()} | {ok, State :: term(), hibernate} | {error, Reason :: term()}.

Initialize the event handler.

Whenever a new event handler is added to an event manager, this function is called to initialize the event handler.

If the event handler is added because of a call to add_handler/3 or add_sup_handler/3, InitArgs is the Args argument of these functions.

If the event handler replaces another event handler because of a call to swap_handler/3 or swap_sup_handler/3, or because of a swap return tuple from one of the other callback functions, InitArgs is a tuple {Args, Term}, where Args is the argument provided in the function call/return tuple and Term is the result of terminating the old event handler, see swap_handler/3.

If successful, the function returns {ok, State} or {ok, State, hibernate}, where State is the initial internal state of the event handler.

If {ok, State, hibernate} is returned, the event manager goes into hibernation (by calling proc_lib:hibernate/3), waiting for the next event to occur.

terminate(Args, State)

(optional)

-callback terminate(Args ::
                        term() |
                        {stop, Reason :: term()} |
                        stop | remove_handler |
                        {error, {'EXIT', Reason :: term()}} |
                        {error, term()},
                    State :: term()) ->
                       term().

Handle event handler termination.

Whenever an event handler is deleted from an event manager, this function is called. It is to be the opposite of Module:init/1 and do any necessary cleaning up.

If the event handler is deleted because of a call to delete_handler/3, swap_handler/3, or swap_sup_handler/3, Arg is the Args argument of this function call.

Arg = {stop, Reason} if the event handler has a supervised connection to a process that has terminated with reason Reason.

Arg = stop if the event handler is deleted because the event manager is terminating.

The event manager terminates if it is part of a supervision tree and it is ordered by its supervisor to terminate. Even if it is not part of a supervision tree, it terminates if it receives an 'EXIT' message from its parent.

Arg = remove_handler if the event handler is deleted because another callback function has returned remove_handler or {remove_handler, Reply}.

Arg = {error, Term} if the event handler is deleted because a callback function returned an unexpected value Term, or Arg = {error, {'EXIT', Reason}} if a callback function failed.

State is the internal state of the event handler.

The function can return any term. If the event handler is deleted because of a call to gen_event:delete_handler/3, the return value of that function becomes the return value of this function. If the event handler is to be replaced with another event handler because of a swap, the return value is passed to the init function of the new event handler. Otherwise the return value is ignored.

Note

This callback is optional, so callback modules need not export it. The gen_event module provides a default implementation without cleanup.

Functions

add_handler(EventMgrRef, Handler, Args)

-spec add_handler(EventMgrRef :: emgr_ref(), Handler :: handler(), Args :: term()) -> term().

Add a new event handler to an event manager.

The new event handler is added to event manager EventMgrRef. The event manager calls Module:init/1 to initiate the event handler and its internal state.

Handler is the name of the callback module Module or a tuple {Module, Id}, where Id is any term. The {Module, Id} representation makes it possible to identify a specific event handler, when many event handlers use the same callback module.

Args is any term that is passed as the argument to Module:init/1.

If Module:init/1 returns a correct value indicating successful completion, the event manager adds the event handler and this function returns ok. If Module:init/1 fails with Reason or returns {error,Reason}, the event handler is ignored and this function returns {'EXIT',Reason} or {error,Reason}, respectively.

add_sup_handler(EventMgrRef, Handler, Args)

-spec add_sup_handler(EventMgrRef :: emgr_ref(), Handler :: handler(), Args :: term()) -> term().

Add a new event handler to an event manager, supervised.

The new event handler is added as for add_handler/3, but the event manager also supervises the connection by linking the event handler and the calling process.

If the calling process later terminates with Reason, the event manager deletes any supervised event handlers by calling Module:terminate/2, then calls Module:handle_info/2 for each remaining handler.

If the event handler is deleted later, the event manager sends a message {gen_event_EXIT,Handler,Reason} to the calling process. Reason is one of the following:

normal, if the event handler has been removed because of a call to delete_handler/3, or remove_handler has been returned by a callback function (see below).

shutdown, if the event handler has been removed because the event manager is terminating.

{swapped, NewHandler, Pid}, if the process Pid has replaced the event handler with another event handler NewHandler, through a call to swap_handler/3 or swap_sup_handler/3.

Other term/0, if the event handler is removed because of an error. Which term depends on the error.

For a description of the arguments and return values, see add_handler/3.

call(EventMgrRef, Handler, Request)

-spec call(EventMgrRef :: emgr_ref(), Handler :: handler(), Request :: term()) -> term().

Equivalent to call(EventMgrRef, Handler, Request, 5000).

call(EventMgrRef, Handler, Request, Timeout)

-spec call(EventMgrRef :: emgr_ref(), Handler :: handler(), Request :: term(), Timeout :: timeout()) ->
              term().

Make a synchronous call to an event handler.

The call is sent to Handler, installed in event manager EventMgrRef, by sending a request and waiting until a reply arrives, or a time-out occurs. The event manager calls Module:handle_call/2 to handle the request.

Request is any term that is passed as one of the arguments to Module:handle_call/2.

Timeout is an integer greater than zero that specifies how many milliseconds to wait for a reply, or the atom infinity to wait indefinitely. Defaults to 5000. If no reply is received within the specified time, the function call fails.

The return value Reply is defined in the return value of Module:handle_call/2. If the specified event handler is not installed, the function returns {error, bad_module}. If the callback function fails with Reason, or returns an unexpected value Term, this function returns {error, {'EXIT', Reason}} or {error, Term}, respectively.

When this call fails it exits the calling process. The exit term is of the form {Reason, Location} where Location = {gen_event, call, ArgList}. See gen_server:call/3 that has a description of relevant values for the Reason in the exit term.

check_response(Msg, ReqId)

(since OTP 23.0)

-spec check_response(Msg, ReqId) -> Result
                        when
                            Msg :: term(),
                            ReqId :: request_id(),
                            Response ::
                                {reply, Reply :: term()} | {error, {Reason :: term(), emgr_ref()}},
                            Result :: Response | no_reply.

Check if a received message is a request response.

Check if Msg is a response corresponding to the request identifier ReqId. The request must have been made by send_request/3, and by the same process calling this function.

If Msg is a response corresponding to ReqId the response is returned in Reply. Otherwise this function returns no_reply and no cleanup is done. Thus this function must be invoked repeatedly until a response is returned.

If the specified event handler is not installed, the function returns {error, bad_module}. If the callback function fails with Reason or returns an unexpected value Term, this function returns {error, {'EXIT', Reason}} or {error, Term}, respectively. If the event manager has died before this function is called, that is; Msg reports the server's death, this function returns {error,{Reason, EventMgrRef}} where Reason is the exit reason.

check_response(Msg, ReqIdCollection, Delete)

(since OTP 25.0)

-spec check_response(Msg, ReqIdCollection, Delete) -> Result
                        when
                            Msg :: term(),
                            ReqIdCollection :: request_id_collection(),
                            Delete :: boolean(),
                            Response ::
                                {reply, Reply :: term()} | {error, {Reason :: term(), emgr_ref()}},
                            Result ::
                                {Response,
                                 Label :: term(),
                                 NewReqIdCollection :: request_id_collection()} |
                                no_request | no_reply.

Check if a received message is a request response in a collection.

Check if Msg is a response corresponding to a request identifier stored in ReqIdCollection. All request identifiers of ReqIdCollection must correspond to requests that have been made using send_request/3 or send_request/5, and all requests must have been made by the process calling this function.

The Label in the response is the Label associated with the request identifier that the response corresponds to. The Label of a request identifier is associated when storing the request id in a collection, or when sending the request using send_request/5.

Compared to check_response/2, the returned result or exception associated with a specific request identifier will be wrapped in a 3-tuple {Response, Label, NewReqIdCollection}. Response is the value that would have been produced by check_response/2, Label is the value associated with the specific request identifier and NewReqIdCollection is a possibly modified request identifier collection.

If ReqIdCollection is empty, no_request will be returned.

If Msg does not correspond to any of the request identifiers in ReqIdCollection, no_reply is returned.

If Delete is true, the association with Label has been deleted from ReqIdCollection in the resulting NewReqIdCollection. If Delete is false, NewReqIdCollection will equal ReqIdCollection. Note that deleting an association is not for free and that a collection containing already handled requests can still be used by subsequent calls to check_response/3), receive_response/3, and wait_response/3.

However, without deleting handled associations, the above calls will not be able to detect when there are no more outstanding requests to handle, so you will have to keep track of this some other way than relying on a no_request return. Note that if you pass a collection only containing associations of already handled or abandoned requests to check_response/3, it will always return no_reply.

delete_handler(EventMgrRef, Handler, Args)

-spec delete_handler(EventMgrRef :: emgr_ref(), Handler :: handler(), Args :: term()) -> term().

Deletes an event handler from an event manager.

This function deletes event handler Handler from event manager EventMgrRef. The event manager calls Module:terminate/2 to terminate the event handler.

Args is any term that is passed as one of the arguments to Module:terminate/2.

The return value is the return value of Module:terminate/2. If the specified event handler is not installed, the function returns {error, module_not_found}. If the callback function fails with Reason, the function returns {'EXIT', Reason}.

notify(EventMgrRef, Event)

-spec notify(EventMgrRef :: emgr_ref(), Event :: term()) -> ok.

Send an asynchronous event notification to an event manager.

The event is sent to EventMgrRef, that calls Module:handle_event/2 for each installed event handler to handle the event.

Event is any term that is passed as one of the arguments to Module:handle_event/2.

notify/1 does not fail even if the specified event manager does not exist, unless it is specified as Name.

receive_response(ReqId, Timeout)

(since OTP 24.0)

-spec receive_response(ReqId, Timeout) -> Result
                          when
                              ReqId :: request_id(),
                              Timeout :: response_timeout(),
                              Response ::
                                  {reply, Reply :: term()} | {error, {Reason :: term(), emgr_ref()}},
                              Result :: Response | timeout.

Receive a request response.

Receive a response corresponding to the request identifier ReqId. The request must have been made by send_request/3, and it must have been made from the same process calling this function.

Timeout specifies how long to wait for a response. If no response is received within the specified time, this function returns timeout. Assuming that the server executes on a node supporting aliases (introduced in OTP 24) the request will also be abandoned. That is, no response will be received after a timeout. Otherwise, a stray response might be received at a later time.

The return value Reply is defined in the return value of Module:handle_call/2.

If the specified event handler is not installed, this function returns {error, bad_module}. If the callback function fails with Reason or returns an unexpected value Term, this function returns {error, {'EXIT', Reason}} or{error,Term}, respectively. If the event manager dies before or during the request this function returns {error, {Reason, EventMgrRef}}.

The difference between wait_response/2 and receive_response/2 is that receive_response/2 abandons the request at time-out so that a potential future response is ignored, while wait_response/2 does not.

receive_response(ReqIdCollection, Timeout, Delete)

(since OTP 25.0)

-spec receive_response(ReqIdCollection, Timeout, Delete) -> Result
                          when
                              ReqIdCollection :: request_id_collection(),
                              Timeout :: response_timeout(),
                              Delete :: boolean(),
                              Response ::
                                  {reply, Reply :: term()} | {error, {Reason :: term(), emgr_ref()}},
                              Result ::
                                  {Response,
                                   Label :: term(),
                                   NewReqIdCollection :: request_id_collection()} |
                                  no_request | timeout.

Receive a request response in a collection.

Receive a response in ReqIdCollection. All request identifiers of ReqIdCollection must correspond to requests that have been made using send_request/3 or send_request/5, and all requests must have been made by the process calling this function.

The Label in the response is the Label associated with the request identifier that the response corresponds to. The Label of a request identifier is associated when adding the request id to a collection, or when sending the request using send_request/5.

Compared to receive_response/2, the returned result or exception associated with a specific request identifier will be wrapped in a 3-tuple {Response, Label, NewReqIdCollection}. Response is the value that would have been produced by receive_response/2, Label is the value associated with the specific request identifier and NewReqIdCollection is a possibly modified request identifier collection.

If ReqIdCollection is empty, no_request will be returned.

Timeout specifies how long to wait for a response. If no response is received within the specified time, the function returns timeout. Assuming that the server executes on a node supporting aliases (introduced in OTP 24) all requests identified by ReqIdCollection will also be abandoned. That is, no responses will be received after a time-out. Otherwise, stray responses might be received at a later time.

The difference between receive_response/3 and wait_response/3 is that receive_response/3 abandons the requests at time-out so that potential future responses are ignored, while wait_response/3 does not.

If Delete is true, the association with Label is deleted from ReqIdCollection in the resulting NewReqIdCollection. If Delete is false, NewReqIdCollection will equal ReqIdCollection. Note that deleting an association is not for free and that a collection containing already handled requests can still be used by subsequent calls to receive_response/3, check_response/3, and wait_response/3.

However, without deleting handled associations, the above calls will not be able to detect when there are no more outstanding requests to handle, so you will have to keep track of this some other way than relying on a no_request return. Note that if you pass a collection only containing associations of already handled or abandoned requests to receive_response/3, it will always block until Timeout expires and then return timeout.

reqids_add(ReqId, Label, ReqIdCollection)

(since OTP 25.0)

-spec reqids_add(ReqId :: request_id(), Label :: term(), ReqIdCollection :: request_id_collection()) ->
                    NewReqIdCollection :: request_id_collection().

Store a request identifier in a colletion.

Stores ReqId and associates a Label with the request identifier by adding this information to ReqIdCollection and returning the resulting request identifier collection.

reqids_new()

(since OTP 25.0)

-spec reqids_new() -> NewReqIdCollection :: request_id_collection().

Create an empty request identifier collection.

Returns a new empty request identifier collection. A request identifier collection can be utilized to handle multiple outstanding requests.

Request identifiers of requests made by send_request/3 can be saved in a request identifier collection using reqids_add/3. Such a collection of request identifiers can later be used in order to get one response corresponding to a request in the collection by passing the collection as argument to receive_response/3, wait_response/3, or, check_response/3.

reqids_size/1 can be used to determine the number of request identifiers in a collection.

reqids_size(ReqIdCollection)

(since OTP 25.0)

-spec reqids_size(ReqIdCollection :: request_id_collection()) -> non_neg_integer().

Returns the number of request identifiers in ReqIdCollection.

reqids_to_list(ReqIdCollection)

(since OTP 25.0)

-spec reqids_to_list(ReqIdCollection :: request_id_collection()) ->
                        [{ReqId :: request_id(), Label :: term()}].

Convert a request identifier collection to a list.

Returns a list of {ReqId, Label} tuples which corresponds to all request identifiers with their associated labels in ReqIdCollection.

send_request(EventMgrRef, Handler, Request)

(since OTP 23.0)

-spec send_request(EventMgrRef :: emgr_ref(), Handler :: handler(), Request :: term()) ->
                      ReqId :: request_id().

Send an asynchronous call request to an event handler.

This function sends the call request Request to the event handler Handler installed in the event manager identified by EventMgrRef, and returns a request identifier ReqId. The return value ReqId shall later be used with receive_response/2, wait_response/2, or check_response/2 to fetch the actual result of the request.

Besides passing the request identifier directly to these functions, it can also be stored in a request identifier collection using reqids_add/3. Such a collection of request identifiers can later be used in order to get one response corresponding to a request in the collection by passing the collection as argument to receive_response/3, wait_response/3, or check_response/3. If you are about to store the request identifier in a collection, you may want to consider using send_request/5 instead.

The calls gen_event:receive_response(gen_event:send_request(EventMgrRef, Handler, Request), Timeout) can be seen as equivalent to gen_event:call(EventMgrRef, Handler, Request, Timeout), ignoring the error handling.

The event manager calls Module:handle_call/2 to handle the request.

Request may be any term and is passed as one of the arguments to Module:handle_call/2.

send_request(EventMgrRef, Handler, Request, Label, ReqIdCollection)

(since OTP 25.0)

-spec send_request(EventMgrRef :: emgr_ref(),
                   Handler :: handler(),
                   Request :: term(),
                   Label :: term(),
                   ReqIdCollection :: request_id_collection()) ->
                      NewReqIdCollection :: request_id_collection().

Send an asynchronous call request to an event handler, storing it in a request identifier collection.

This function sends the call request Request to the event handler Handler installed in the event manager identified by EventMgrRef. The Label will be associated with the request identifier of the operation and added to the returned request identifier collection NewReqIdCollection.

The collection can later be used in order to get one response corresponding to a request in the collection by passing the collection as argument to receive_response/3, wait_response/3, or check_response/3.

The same as calling gen_event:reqids_add(gen_event:send_request(EventMgrRef, Handler, Request), Label, ReqIdCollection), but slightly more efficient.

start()

-spec start() -> start_ret().

Equivalent to start([]).

start/1

-spec start(EventMgrName :: emgr_name()) -> start_ret();
           (Options :: options()) -> start_ret().

Create a stand-alone event manager process, possibly nameless.

Equivalent to start(EventMgrName, Options).

With argument EventMgrName, Options is [].

With argument Options a nameless event manager is created.

For a description of the arguments and return values, see start_link/2.

start(EventMgrName, Options)

(since OTP 20.0)

-spec start(EventMgrName :: emgr_name(), Options :: options()) -> start_ret().

Create a stand-alone event manager process.

The created event manager process is not part of a supervision tree and thus has no supervisor.

For a description of the arguments and return values, see start_link/2.

start_link()

-spec start_link() -> start_ret().

Equivalent to start_link([]).

start_link/1

-spec start_link(EventMgrName :: emgr_name()) -> start_ret();
                (Options :: options()) -> start_ret().

Create an event manager process as part of a supervision tree, possibly nameless.

Equivalent to start_link(EventMgrName, Options).

With argument EventMgrName, Options is [].

With argument Options a nameless event manager is created.

For a description of the arguments and return values, see start_link/2.

start_link(EventMgrName, Options)

(since OTP 20.0)

-spec start_link(EventMgrName :: emgr_name(), Options :: options()) -> start_ret().

Create an event manager process as part of a supervision tree.

The function is to be called, directly or indirectly, by the supervisor. For example, it ensures that the event manager is linked to the caller (supervisor).

If option {hibernate_after, HibernateAfterTimeout} is present, the gen_event process awaits any message for HibernateAfterTimeout milliseconds and if no message is received, the process goes into hibernation automatically (by calling proc_lib:hibernate/3).

If the event manager is successfully created, the function returns {ok, Pid} where Pid is the pid/0 of the event manager.

If a process with the specified EventMgrName exists already, the function returns {error,{already_started,OtherPid}}, where OtherPid is the pid of that process, and the event manager process exits with reason normal.

If the event manager fails to start within the specified start timeout {timeout, Time}, which is very unlikely since the start does not interact with other processes, the function returns {error, timeout} and the failed event manager is killed with exit(_, kill).

If start_link/1,2 returns {error, _}, the started event manager process has terminated. If an 'EXIT' message was delivered to the calling process (due to the process link), that message has been consumed.

Warning

Before OTP 26.0, if the started event manager failed to register its name, this founction could return {error, {already_started, OtherPid}} before the started event manager process had terminated, so starting again might fail because the registered name was not yet unregistered, and an 'EXIT' message could arrive later to the process calling this function.

But if the start timed out, this function killed the started event manager process and returned {error, timeout}, and then the process link {'EXIT', Pid, killed} message was consumed.

The start was made synchronous in OTP 26.0 and a guarantee was implemented that no process link 'EXIT' message from a failed start will linger in the caller's inbox.

start_monitor()

(since OTP 23.0)

-spec start_monitor() -> start_mon_ret().

Equivalent to start_monitor([]).

start_monitor(EventMgrNameOrOptions)

(since OTP 23.0)

-spec start_monitor(EventMgrNameOrOptions :: emgr_name() | options()) -> start_mon_ret().

Creates a stand-alone event manager process, monitored, possibly nameless.

Equivalent to start_monitor(EventMgrName, Options).

With argument EventMgrName, Options is [].

With argument Options a nameless event manager is created.

For a description of the arguments and return values, see start_monitor/2 and start_link/1.

start_monitor(EventMgtName, Options)

(since OTP 23.0)

-spec start_monitor(EventMgtName :: emgr_name(), Options :: options()) -> start_mon_ret().

Creates a stand-alone event manager process, monitored.

The created event manager process is not part of a supervision tree and thus has no supervisor. A monitor is atomically set up to the newly created process.

For a description of the arguments and return values, see start_link/2. Note that the return value for a successful start differs from start_link/2. start_monitor/0,1,2 will return {ok, {Pid, Mon}} where Pid is the process identifier of the process, and Mon is a reference to the monitor set up to monitor the process. If the start is not successful, the caller will be blocked until the DOWN message has been received and removed from the message queue.

stop(EventMgrRef)

-spec stop(EventMgrRef :: emgr_ref()) -> ok.

Equivalent to stop(EventMgrRef, normal, infinity).

stop(EventMgrRef, Reason, Timeout)

(since OTP 18.0)

-spec stop(EventMgrRef :: emgr_ref(), Reason :: term(), Timeout :: timeout()) -> ok.

Stop an event manager.

Orders event manager EventMgrRef to exit with the specifies Reason, and waits for it to terminate. Before terminating, gen_event calls Module:terminate(stop,...) for each installed event handler.

The function returns ok if the event manager terminates with the expected reason. Any other reason than normal, shutdown, or {shutdown, Term} causes an error report to be issued using logger.

Timeout is an integer greater than zero that specifies how many milliseconds to wait for the event manager to terminate, or the atom infinity to wait indefinitely. If the event manager has not terminated within the specified time, the call exits the calling process with reason timeout.

If the process does not exist, the call exits the calling process with reason noproc, and with reason {nodedown, Node} if the connection fails to the remote Node where the server runs.

swap_handler(EventMgrRef, OldHandler, NewHandler)

-spec swap_handler(EventMgrRef :: emgr_ref(),
                   OldHandler :: {handler(), term()},
                   NewHandler :: {handler(), term()}) ->
                      ok | {error, term()}.

Replace an event handler.

This function replaces an event handler in event manager EventMgrRef.

For a description of OldHandler and NewHandler, see add_handler/3.

First the old event handler OldHandler is deleted. The event manager calls OldModule:terminate(Args1, ...), where OldModule is the callback module of OldHandler, and collects the return value.

Then the new event handler NewHandler is added and initiated by calling NewModule:init({Args2,Term}), where NewModule is the callback module of NewHandler, and Term is the return value of OldModule:terminate/2. This makes it possible to transfer information from OldHandler to NewHandler.

The new handler is added even if the the specified old event handler is not installed, in which case Term = error, or if OldModule:terminate/2 fails with Reason, in which case Term = {'EXIT', Reason}. The old handler is deleted even if NewModule:init/1 fails.

If there was a supervised connection between OldHandler and a process Pid, there is a supervised connection between NewHandler and Pid instead.

If NewModule:init/1 returns a correct value, this function returns ok. If NewModule:init/1 fails with Reason or returns an unexpected value Term, this function returns {error, {'EXIT', Reason}} or {error, Term}, respectively.

swap_sup_handler(EventMgrRef, OldHandler, NewHandler)

-spec swap_sup_handler(EventMgrRef :: emgr_ref(),
                       OldHandler :: {handler(), term()},
                       NewHandler :: {handler(), term()}) ->
                          ok | {error, term()}.

Replace an event handler, and supervise it.

Replaces an event handler in event manager EventMgrRef in the same way as swap_handler/3, but also supervises the connection between NewHandler and the calling process.

For a description of the arguments and return values, see swap_handler/3.

sync_notify(EventMgrRef, Event)

-spec sync_notify(EventMgrRef :: emgr_ref(), Event :: term()) -> ok.

Send a synchronous event notification to an event manager.

The event is sent to EventMgrRef that calls Module:handle_event/2 for each installed event handler to handle the event. This function will return ok after the event has been handled by all event handlers.

Event is any term that is passed as one of the arguments to Module:handle_event/2.

wait_response(ReqId, WaitTime)

(since OTP 23.0)

-spec wait_response(ReqId, WaitTime) -> Result
                       when
                           ReqId :: request_id(),
                           WaitTime :: response_timeout(),
                           Response ::
                               {reply, Reply :: term()} | {error, {Reason :: term(), emgr_ref()}},
                           Result :: Response | timeout.

Wait for a request resonse.

Wait for the response to the request identifier ReqId. The request must have been made by send_request/3, from the same process that called send_request/3.

WaitTime specifies how long to wait for a response. If no response is received within the specified time, the function returns timeout and no cleanup is done, Thus the function can be invoked repeatedly until a reply is returned.

The return value Reply is defined in the return value of Module:handle_call/2.

If the specified event handler is not installed, the function returns {error, bad_module}. If the callback function fails with Reason, or returns an unexpected value Term, this function returns {error,{'EXIT',Reason}} or {error,Term}, respectively. If the event manager dies before or during the request this function returns {error, {Reason, EventMgrRef}}.

The difference between receive_response/2 and wait_response/2 is that receive_response/2 abandons the request at timeout so that a potential future response is ignored, while wait_response/2 does not.

wait_response(ReqIdCollection, WaitTime, Delete)

(since OTP 25.0)

-spec wait_response(ReqIdCollection, WaitTime, Delete) -> Result
                       when
                           ReqIdCollection :: request_id_collection(),
                           WaitTime :: response_timeout(),
                           Delete :: boolean(),
                           Response ::
                               {reply, Reply :: term()} | {error, {Reason :: term(), emgr_ref()}},
                           Result ::
                               {Response,
                                Label :: term(),
                                NewReqIdCollection :: request_id_collection()} |
                               no_request | timeout.

Wait for any request response in a collection.

Wait for a response in a ReqIdCollection. All request identifiers of ReqIdCollection must correspond to requests that have been made using send_request/3 or send_request/5, and all requests must have been made by the process calling this function.

The Label in the response is the Label associated with the request identifier that the response corresponds to. The Label of a request identifier is associated when adding the request id to a collection, or when sending the request using send_request/5.

Compared to wait_response/2, the returned result or exception associated with a specific request identifier will be wrapped in a 3-tuple {Response, Label, NewReqIdCollection}. Response is the value that would have been produced by wait_response/2, Label is the value associated with the specific request identifier and NewReqIdCollection is a possibly modified request identifier collection.

If ReqIdCollection is empty, no_request will be returned.

If no response is received before the WaitTime has expired, timeout is returned. It is valid to continue waiting for a response as many times as needed up until a response has been received and completed by check_response(), receive_response(), or wait_response().

The difference between receive_response/3 and wait_response/3 is that receive_response/3 abandons requests at time-out so that potential future responses are ignored, while wait_response/3 does not.

If Delete is true, the association with Label has been deleted from ReqIdCollection in the resulting NewReqIdCollection. If Delete isfalse, NewReqIdCollection will equal ReqIdCollection. Note that deleting an association is not for free and that a collection containing already handled requests can still be used by subsequent calls to wait_response/3, check_response/3, and receive_response/3.

However, without deleting handled associations, the above calls will not be able to detect when there are no more outstanding requests to handle, so you will have to keep track of this some other way than relying on a no_request return. Note that if you pass a collection only containing associations of already handled or abandoned requests to this function, it will always block until WaitTime expires and then return timeout.

which_handlers(EventMgrRef)

-spec which_handlers(EventMgrRef :: emgr_ref()) -> [handler()].

Return all event handlers in an event manager.

This function returns a list of all event handlers installed in event manager EventMgrRef.

For a description of Handler, see add_handler/3.

gen_fsm behaviour

Deprecated and replaced by gen_statem in OTP 20.

Migration to gen_statem

Here follows a simple example of turning a gen_fsm into a gen_statem. The example comes from the previous User's Guide for gen_fsm

-module(code_lock).
-define(NAME, code_lock).
%-define(BEFORE_REWRITE, true).

-ifdef(BEFORE_REWRITE).
-behaviour(gen_fsm).
-else.
-behaviour(gen_statem).
-endif.

-export([start_link/1, button/1, stop/0]).

-ifdef(BEFORE_REWRITE).
-export([init/1, locked/2, open/2, handle_sync_event/4, handle_event/3,
     handle_info/3, terminate/3, code_change/4]).
-else.
-export([init/1, callback_mode/0, locked/3, open/3,
     terminate/3, code_change/4]).
%% Add callback__mode/0
%% Change arity of the state functions
%% Remove handle_info/3
-endif.

-ifdef(BEFORE_REWRITE).
start_link(Code) ->
    gen_fsm:start_link({local, ?NAME}, ?MODULE, Code, []).
-else.
start_link(Code) ->
    gen_statem:start_link({local,?NAME}, ?MODULE, Code, []).
-endif.

-ifdef(BEFORE_REWRITE).
button(Digit) ->
    gen_fsm:send_event(?NAME, {button, Digit}).
-else.
button(Digit) ->
    gen_statem:cast(?NAME, {button,Digit}).
    %% send_event is asynchronous and becomes a cast
-endif.

-ifdef(BEFORE_REWRITE).
stop() ->
    gen_fsm:sync_send_all_state_event(?NAME, stop).
-else.
stop() ->
    gen_statem:call(?NAME, stop).
    %% sync_send is synchronous and becomes call
    %% all_state is handled by callback code in gen_statem
-endif.

init(Code) ->
    do_lock(),
    Data = #{code => Code, remaining => Code},
    {ok, locked, Data}.

-ifdef(BEFORE_REWRITE).
-else.
callback_mode() ->
    state_functions.
%% state_functions mode is the mode most similar to
%% gen_fsm. There is also handle_event mode which is
%% a fairly different concept.
-endif.

-ifdef(BEFORE_REWRITE).
locked({button, Digit}, Data0) ->
    case analyze_lock(Digit, Data0) of
    {open = StateName, Data} ->
        {next_state, StateName, Data, 10000};
    {StateName, Data} ->
        {next_state, StateName, Data}
    end.
-else.
locked(cast, {button,Digit}, Data0) ->
    case analyze_lock(Digit, Data0) of
    {open = StateName, Data} ->
        {next_state, StateName, Data, 10000};
    {StateName, Data} ->
        {next_state, StateName, Data}
    end;
locked({call, From}, Msg, Data) ->
    handle_call(From, Msg, Data);
locked({info, Msg}, StateName, Data) ->
    handle_info(Msg, StateName, Data).
%% Arity differs
%% All state events are dispatched to handle_call and handle_info help
%% functions. If you want to handle a call or cast event specifically
%% for this state you would add a special clause for it above.
-endif.

-ifdef(BEFORE_REWRITE).
open(timeout, State) ->
     do_lock(),
    {next_state, locked, State};
open({button,_}, Data) ->
    {next_state, locked, Data}.
-else.
open(timeout, _, Data) ->
    do_lock(),
    {next_state, locked, Data};
open(cast, {button,_}, Data) ->
    {next_state, locked, Data};
open({call, From}, Msg, Data) ->
    handle_call(From, Msg, Data);
open(info, Msg, Data) ->
    handle_info(Msg, open, Data).
%% Arity differs
%% All state events are dispatched to handle_call and handle_info help
%% functions. If you want to handle a call or cast event specifically
%% for this state you would add a special clause for it above.
-endif.

-ifdef(BEFORE_REWRITE).
handle_sync_event(stop, _From, _StateName, Data) ->
    {stop, normal, ok, Data}.

handle_event(Event, StateName, Data) ->
    {stop, {shutdown, {unexpected, Event, StateName}}, Data}.

handle_info(Info, StateName, Data) ->
    {stop, {shutdown, {unexpected, Info, StateName}}, StateName, Data}.
-else.
-endif.

terminate(_Reason, State, _Data) ->
    State =/= locked andalso do_lock(),
    ok.
code_change(_Vsn, State, Data, _Extra) ->
    {ok, State, Data}.

%% Internal functions
-ifdef(BEFORE_REWRITE).
-else.
handle_call(From, stop, Data) ->
     {stop_and_reply, normal,  {reply, From, ok}, Data}.

handle_info(Info, StateName, Data) ->
    {stop, {shutdown, {unexpected, Info, StateName}}, StateName, Data}.
%% These are internal functions for handling all state events
%% and not behaviour callbacks as in gen_fsm
-endif.

analyze_lock(Digit, #{code := Code, remaining := Remaining} = Data) ->
     case Remaining of
         [Digit] ->
         do_unlock(),
         {open,  Data#{remaining := Code}};
         [Digit|Rest] -> % Incomplete
             {locked, Data#{remaining := Rest}};
         _Wrong ->
             {locked, Data#{remaining := Code}}
     end.

do_lock() ->
    io:format("Lock~n", []).
do_unlock() ->
    io:format("Unlock~n", []).

OTP 19 Documentation

Module

gen_fsm

Module Summary

Generic finite state machine behavior.

Description

This behavior module provides a finite state machine. A generic finite state machine process (gen_fsm) implemented using this module has a standard set of interface functions and includes functionality for tracing and error reporting. It also fits into an OTP supervision tree. For more information, see OTP Design Principles.

A gen_fsm process assumes all specific parts to be located in a callback module exporting a predefined set of functions. The relationship between the behavior functions and the callback functions is as follows:

gen_fsm module                    Callback module
--------------                    ---------------
gen_fsm:start
gen_fsm:start_link                -----> Module:init/1

gen_fsm:stop                      -----> Module:terminate/3

gen_fsm:send_event                -----> Module:StateName/2

gen_fsm:send_all_state_event      -----> Module:handle_event/3

gen_fsm:sync_send_event           -----> Module:StateName/3

gen_fsm:sync_send_all_state_event -----> Module:handle_sync_event/4

-                                 -----> Module:handle_info/3

-                                 -----> Module:terminate/3

-                                 -----> Module:code_change/4

If a callback function fails or returns a bad value, the gen_fsm process terminates.

A gen_fsm process handles system messages as described in sys(3). The sys module can be used for debugging a gen_fsm process.

Notice that a gen_fsm process does not trap exit signals automatically, this must be explicitly initiated in the callback module.

Unless otherwise stated, all functions in this module fail if the specified gen_fsm process does not exist or if bad arguments are specified.

The gen_fsm process can go into hibernation (see erlang:hibernate/3) if a callback function specifies hibernate instead of a time-out value. This can be useful if the server is expected to be idle for a long time. However, use this feature with care, as hibernation implies at least two garbage collections (when hibernating and shortly after waking up) and is not something you want to do between each call to a busy state machine.

Callback Functions

See the Callback Functions section for the functions to be exported from a gen_fsm callback module.

State name denotes a state of the state machine.

State data denotes the internal state of the Erlang process that implements the state machine.

Summary

Types

enter_loop_opt()

Start options for the enter_loop/4,5,6, start/3,4, and start_link/3,4 functions.

from()

Reply destination. See reply/2

fsm_name()

FSM name specification: local, global, or via registered.

fsm_ref()

FSM reference pid/0 or registered fsm_name/0.

start_opt()

Start options for the start/3,4, and start_link/3,4 functions.

Callbacks: deprecated

code_change(OldVsn, StateName, StateData, Extra) deprecated

Update the internal state data during upgrade/downgrade.

format_status/2 deprecated

Optional function for providing a term describing the current gen_fsm process status.

handle_event(Event, StateName, StateData) deprecated

Handle an asynchronous event.

handle_info(Info, StateName, StateData) deprecated

Handle an incoming message

handle_sync_event(Event, From, StateName, StateData) deprecated

Handle a synchronous event.

init(Args) deprecated

Initialize process and internal state name and state data.

'StateName'(Event, StateData) deprecated

Handle an asynchronous event.

'StateName'(Event, From, StateData) deprecated

Handle a synchronous event.

terminate(Reason, StateName, StateData) deprecated

Clean up before termination.

Functions

cancel_timer(Ref) deprecated

Cancel an internal timer in a generic FSM.

enter_loop(Module, Options, StateName, StateData) deprecated

Enter the gen_fsm receive loop.

enter_loop/5 deprecated

Enter the gen_fsm receive loop.

enter_loop(Module, Options, StateName, StateData, FsmName, Timeout) deprecated

Enter the gen_fsm receive loop.

reply(Caller, Reply) deprecated

Send a reply to a caller.

send_all_state_event(FsmRef, Event) deprecated

Send an event asynchronously to a generic FSM.

send_event(FsmRef, Event) deprecated

Send an event asynchronously to a generic FSM.

send_event_after(Time, Event) deprecated

Send a delayed event internally in a generic FSM.

start(Module, Args, Options) deprecated

Create a standalone gen_fsm process, not registered.

start(FsmName, Module, Args, Options) deprecated

Create a standalone gen_fsm process.

start_link(Module, Args, Options) deprecated

Create a gen_fsm process in a supervision tree, not registered.

start_link(FsmName, Module, Args, Options) deprecated

Create a gen_fsm process in a supervision tree.

start_timer(Time, Msg) deprecated

Send a time-out event internally in a generic FSM.

stop(FsmRef) deprecated

Equivalent to stop(FsmRef, normal, infinity).

stop(FsmRef, Reason, Timeout) deprecated

Synchronously stop a generic FSM.

sync_send_all_state_event(FsmRef, Event) deprecated

Equivalent to sync_send_all_state_event(FsmRef, Event, 5000).

sync_send_all_state_event(FsmRef, Event, Timeout) deprecated

Send an event synchronously to a generic FSM.

sync_send_event(FsmRef, Event) deprecated

Equivalent to sync_send_event(FsmRef, Event, 5000).

sync_send_event(FsmRef, Event, Timeout) deprecated

Send an event synchronously to a generic FSM.

Types

enter_loop_opt()

(not exported)

-type enter_loop_opt() :: {debug, Dbgs :: [sys:debug_option()]}.

Start options for the enter_loop/4,5,6, start/3,4, and start_link/3,4 functions.

See start_link/4.

from()

(not exported)

-type from() :: {To :: pid(), Tag :: term()}.

Reply destination. See reply/2

fsm_name()

(not exported)

-type fsm_name() ::
          {local, LocalName :: atom()} |
          {global, GlobalName :: term()} |
          {via, RegMod :: module(), ViaName :: term()}.

FSM name specification: local, global, or via registered.

To be used when starting a gen_fsm. See start_link/4.

fsm_ref()

(not exported)

-type fsm_ref() ::
          pid() |
          (LocalName :: atom()) |
          {Name :: atom(), Node :: atom()} |
          {global, GlobalName :: term()} |
          {via, RegMod :: module(), ViaName :: term()}.

FSM reference pid/0 or registered fsm_name/0.

To be used in for example send_event/2 to specify the server.

start_opt()

(not exported)

-type start_opt() ::
          {timeout, Time :: timeout()} | {spawn_opt, [proc_lib:start_spawn_option()]} | enter_loop_opt().

Start options for the start/3,4, and start_link/3,4 functions.

See start_link/4.

Callbacks: deprecated

code_change(OldVsn, StateName, StateData, Extra)

(optional)

This callback is deprecated. the callback gen_fsm:code_change(_,_,_,_) is deprecated; use the 'gen_statem' module instead.

-callback code_change(OldVsn, StateName, StateData, Extra) -> {ok, NextStateName, NewStateData}
                         when
                             OldVsn :: Vsn | {down, Vsn},
                             Vsn :: term(),
                             StateName :: atom(),
                             NextStateName :: atom(),
                             StateData :: term(),
                             NewStateData :: term(),
                             Extra :: term().

Update the internal state data during upgrade/downgrade.

This function is called by a gen_fsm process when it is to update its internal state data during a release upgrade/downgrade, that is, when instruction {update, Module, Change, ...}, where Change = {advanced, Extra}, is given in the appup file; see Release Handling Instructions in OTP Design Principles.

For an upgrade, OldVsn is Vsn, and for a downgrade, OldVsn is {down, Vsn}. Vsn is defined by the vsn attribute(s) of the old version of the callback module Module. If no such attribute is defined, the version is the checksum of the Beam file.

StateName is the current state name and StateData the internal state data of the gen_fsm process.

Extra is passed "as is" from the {advanced, Extra} part of the update instruction.

The function is to return the new current state name and updated internal data.

format_status/2

(optional)

This callback is deprecated. the callback gen_fsm:format_status(_,_) is deprecated; use the 'gen_statem' module instead.

-callback format_status(Opt, nonempty_improper_list(PDict, [StateData])) -> Status
                           when
                               Opt :: normal | terminate,
                               PDict :: [{Key :: term(), Value :: term()}],
                               StateData :: term(),
                               Status :: term().

Optional function for providing a term describing the current gen_fsm process status.

The second argument is [PDict, StateData], that is, a list with the 2 elements, in that order.

Note

This callback is optional, so callback modules need not export it. The gen_fsm module provides a default implementation of this function that returns the callback module state data.

This function is called by a gen_fsm process in the following situations:

One of sys:get_status/1,2 is invoked to get the gen_fsm status. Opt is set to the atom normal for this case.

The gen_fsm process terminates abnormally and logs an error. Opt is set to the atom terminate for this case.

This function is useful for changing the form and appearance of the gen_fsm status for these cases. A callback module wishing to change the sys:get_status/1,2 return value as well as how its status appears in termination error logs, exports an instance of format_status/2 that returns a term describing the current status of the gen_fsm process.

PDict is the current value of the process dictionary of the gen_fsm process.

StateData is the internal state data of the gen_fsm process.

The function is to return Status, a term that change the details of the current state and status of the gen_fsm process. There are no restrictions of the form Status can take, but for the sys:get_status/1,2 case (when Opt is normal), the recommended form for the Status value is [{data, [{"StateData", Term}]}], where Term provides relevant details of the gen_fsm state data. Following this recommendation is not required, but it makes the callback module status consistent with the rest of the sys:get_status/1,2 return value.

One use for this function is to return compact alternative state data representations to avoid that large state terms are printed in log files.

handle_event(Event, StateName, StateData)

This callback is deprecated. the callback gen_fsm:handle_event(_,_,_) is deprecated; use the 'gen_statem' module instead.

-callback handle_event(Event, StateName, StateData) -> Result
                          when
                              Event :: term(),
                              StateName :: atom(),
                              StateData :: term(),
                              Result ::
                                  {next_state, NextStateName, NewStateData} |
                                  {next_state, NextStateName, NewStateData, Timeout} |
                                  {next_state, NextStateName, NewStateData, hibernate} |
                                  {stop, Reason, NewStateData},
                              NextStateName :: atom(),
                              NewStateData :: term(),
                              Timeout :: timeout(),
                              Reason :: term().

Handle an asynchronous event.

Whenever a gen_fsm process receives an event sent using send_all_state_event/2, this function is called to handle the event.

StateName is the current state name of the gen_fsm process.

For a description of the other arguments and possible return values, see Module:StateName/2.

handle_info(Info, StateName, StateData)

(optional)

This callback is deprecated. the callback gen_fsm:handle_info(_,_,_) is deprecated; use the 'gen_statem' module instead.

-callback handle_info(Info, StateName, StateData) -> Result
                         when
                             Info :: term(),
                             StateName :: atom(),
                             StateData :: term(),
                             Result ::
                                 {next_state, NextStateName, NewStateData} |
                                 {next_state, NextStateName, NewStateData, Timeout} |
                                 {next_state, NextStateName, NewStateData, hibernate} |
                                 {stop, Reason, NewStateData},
                             NextStateName :: atom(),
                             NewStateData :: term(),
                             Timeout :: timeout(),
                             Reason :: normal | term().

Handle an incoming message

This function is called by a gen_fsm process when it receives any other message than a synchronous or asynchronous event (or a system message).

Info is the received message.

For a description of the other arguments and possible return values, see Module:StateName/2.

handle_sync_event(Event, From, StateName, StateData)

This callback is deprecated. the callback gen_fsm:handle_sync_event(_,_,_,_) is deprecated; use the 'gen_statem' module instead.

-callback handle_sync_event(Event, From, StateName, StateData) -> Result
                               when
                                   Event :: term(),
                                   From :: from(),
                                   StateName :: atom(),
                                   StateData :: term(),
                                   Result ::
                                       {reply, Reply, NextStateName, NewStateData} |
                                       {reply, Reply, NextStateName, NewStateData, Timeout} |
                                       {reply, Reply, NextStateName, NewStateData, hibernate} |
                                       {next_state, NextStateName, NewStateData} |
                                       {next_state, NextStateName, NewStateData, Timeout} |
                                       {next_state, NextStateName, NewStateData, hibernate} |
                                       {stop, Reason, Reply, NewStateData} |
                                       {stop, Reason, NewStateData},
                                   Reply :: term(),
                                   NextStateName :: atom(),
                                   NewStateData :: term(),
                                   Timeout :: timeout(),
                                   Reason :: term().

Handle a synchronous event.

Whenever a gen_fsm process receives an event sent using sync_send_all_state_event/2,3, this function is called to handle the event.

StateName is the current state name of the gen_fsm process.

For a description of the other arguments and possible return values, see Module:StateName/3.

init(Args)

This callback is deprecated. the callback gen_fsm:init(_) is deprecated; use the 'gen_statem' module instead.

-callback init(Args) -> Result
                  when
                      Args :: term(),
                      Result ::
                          {ok, StateName, StateData} |
                          {ok, StateName, StateData, Timeout} |
                          {ok, StateName, StateData, hibernate} |
                          {stop, Reason} |
                          ignore,
                      StateName :: atom(),
                      StateData :: term(),
                      Timeout :: timeout(),
                      Reason :: term().

Initialize process and internal state name and state data.

Whenever a gen_fsm process is started using start/3,4 or start_link/3,4, this function is called by the new process to initialize.

Args is the Args argument provided to the start function.

If initialization is successful, the function is to return {ok, StateName, StateData}, {ok, StateName, StateData, Timeout}, or {ok, StateName, StateData, hibernate}, where StateName is the initial state name and StateData the initial state data of the gen_fsm process.

If an integer/0 time-out value is provided, a time-out occurs unless an event or a message is received within Timeout milliseconds. A time-out is represented by the atom timeout and is to be handled by the Module:StateName/2 callback functions. The atom infinity can be used to wait indefinitely, this is the default value.

If hibernate is specified instead of a time-out value, the process goes into hibernation when waiting for the next message to arrive (by calling proc_lib:hibernate/3).

If the initialization fails, the function returns {stop, Reason}, where Reason is any term, or ignore.

'StateName'(Event, StateData)

(optional)

This callback is deprecated. the callback gen_fsm:'StateName'(_,_) is deprecated; use the 'gen_statem' module instead.

-callback 'StateName'(Event, StateData) -> Result
                         when
                             Event :: timeout | term(),
                             StateData :: term(),
                             Result ::
                                 {next_state, NextStateName, NewStateData} |
                                 {next_state, NextStateName, NewStateData, Timeout} |
                                 {next_state, NextStateName, NewStateData, hibernate} |
                                 {stop, Reason, NewStateData},
                             NextStateName :: atom(),
                             NewStateData :: term(),
                             Timeout :: timeout(),
                             Reason :: term().

Handle an asynchronous event.

There is to be one instance of this function for each possible state name. Whenever a gen_fsm process receives an event sent using send_event/2, the instance of this function with the same name as the current state name StateName is called to handle the event. It is also called if a time-out occurs.

Event is either the atom timeout, if a time-out has occurred, or the Event argument provided to send_event/2.

StateData is the state data of the gen_fsm process.

If the function returns {next_state, NextStateName, NewStateData}, {next_state, NextStateName, NewStateData, Timeout}, or {next_state, NextStateName, NewStateData, hibernate}, the gen_fsm process continues executing with the current state name set to NextStateName and with the possibly updated state data NewStateData. For a description of Timeout and hibernate, see Module:init/1.

If the function returns {stop ,Reason, NewStateData}, the gen_fsm process calls Module:terminate(Reason, StateName, NewStateData) and terminates.

'StateName'(Event, From, StateData)

(optional)

This callback is deprecated. the callback gen_fsm:'StateName'(_,_,_) is deprecated; use the 'gen_statem' module instead.

-callback 'StateName'(Event, From, StateData) -> Result
                         when
                             Event :: term(),
                             From :: from(),
                             StateData :: term(),
                             Result ::
                                 {reply, Reply, NextStateName, NewStateData} |
                                 {reply, Reply, NextStateName, NewStateData, Timeout} |
                                 {reply, Reply, NextStateName, NewStateData, hibernate} |
                                 {next_state, NextStateName, NewStateData} |
                                 {next_state, NextStateName, NewStateData, Timeout} |
                                 {next_state, NextStateName, NewStateData, hibernate} |
                                 {stop, Reason, Reply, NewStateData} |
                                 {stop, Reason, NewStateData},
                             Reply :: term(),
                             NextStateName :: atom(),
                             NewStateData :: term(),
                             Timeout :: timeout(),
                             Reason :: normal | term().

Handle a synchronous event.

There is to be one instance of this function for each possible state name. Whenever a gen_fsm process receives an event sent using sync_send_event/2,3, the instance of this function with the same name as the current state name StateName is called to handle the event.

Event is the Event argument provided to sync_send_event/2,3.

From is a tuple {Pid, Tag} where Pid is the pid/0 of the process that called sync_send_event/2,3, Tag is a unique tag.

StateData is the state data of the gen_fsm process.

If {reply, Reply, NextStateName, NewStateData}, {reply, Reply, NextStateName, NewStateData, Timeout}, or {reply, Reply, NextStateName, NewStateData, hibernate} is returned, Reply is given back to From as the return value of sync_send_event/2,3. The gen_fsm process then continues executing with the current state name set to NextStateName and with the possibly updated state data NewStateData. For a description of Timeout and hibernate, see Module:init/1.

If {next_state, NextStateName, NewStateData}, {next_state, NextStateName, NewStateData, Timeout}, or {next_state, NextStateName, NewStateData, hibernate} is returned, the gen_fsm process continues executing in NextStateName with NewStateData. Any reply to From must be specified explicitly using reply/2.

If the function returns {stop, Reason, Reply, NewStateData}, Reply is given back to From. If the function returns {stop, Reason, NewStateData}, any reply to From must be specified explicitly using reply/2. The gen_fsm process then calls Module:terminate(Reason, StateName, NewStateData) and terminates.

terminate(Reason, StateName, StateData)

(optional)

This callback is deprecated. the callback gen_fsm:terminate(_,_,_) is deprecated; use the 'gen_statem' module instead.

-callback terminate(Reason, StateName, StateData) -> _
                       when
                           Reason :: normal | shutdown | {shutdown, term()} | term(),
                           StateName :: atom(),
                           StateData :: term().

Clean up before termination.

This function is called by a gen_fsm process when it is about to terminate. It is to be the opposite of Module:init/1 and do any necessary cleaning up. When it returns, the gen_fsm process terminates with Reason. The return value is ignored.

Reason is a term denoting the stop reason, StateName is the current state name, and StateData is the state data of the gen_fsm process.

Reason depends on why the gen_fsm process is terminating. If it is because another callback function has returned a stop tuple {stop, ...}, Reason has the value specified in that tuple. If it is because of a failure, Reason is the error reason.

If the gen_fsm process is part of a supervision tree and is ordered by its supervisor to terminate, this function is called with Reason = shutdown if the following conditions apply:

The gen_fsm process has been set to trap exit signals.

The shutdown strategy as defined in the child specification of the supervisor is an integer time-out value, not brutal_kill.

Even if the gen_fsm process is not part of a supervision tree, this function is called if it receives an 'EXIT' message from its parent. Reason is the same as in the 'EXIT' message.

Otherwise, the gen_fsm process terminates immediately.

Notice that for any other reason than normal, shutdown, or {shutdown, Term} the gen_fsm process is assumed to terminate because of an error and an error report is issued using error_logger:format/2.

Functions

cancel_timer(Ref)

This function is deprecated. gen_fsm:cancel_timer/1 is deprecated; use the 'gen_statem' module instead.

-spec cancel_timer(Ref) -> RemainingTime | false
                      when Ref :: reference(), RemainingTime :: non_neg_integer().

Cancel an internal timer in a generic FSM.

Cancels an internal timer referred by Ref in the gen_fsm process that calls this function.

Ref is a reference returned from send_event_after/2 or start_timer/2.

If the timer has already timed out, but the event not yet been delivered, it is cancelled as if it had not timed out, so there is no false timer event after returning from this function.

Returns the remaining time in milliseconds until the timer would have expired if Ref referred to an active timer, otherwise false.

enter_loop(Module, Options, StateName, StateData)

This function is deprecated. gen_fsm:enter_loop/4 is deprecated; use the 'gen_statem' module instead.

-spec enter_loop(Module, Options, StateName, StateData) -> no_return()
                    when
                        Module :: module(),
                        Options :: [enter_loop_opt()],
                        StateName :: atom(),
                        StateData :: term().

Enter the gen_fsm receive loop.

Equivalent to enter_loop/6 with Timeout = infinity but the started server is not registered as for start_link/3.

enter_loop/5

This function is deprecated. gen_fsm:enter_loop/5 is deprecated; use the 'gen_statem' module instead.

-spec enter_loop(Module, Options, StateName, StateData, FsmName) -> no_return()
                    when
                        Module :: module(),
                        Options :: [enter_loop_opt()],
                        StateName :: atom(),
                        StateData :: term(),
                        FsmName :: fsm_name();
                (Module, Options, StateName, StateData, Timeout) -> no_return()
                    when
                        Module :: module(),
                        Options :: enter_loop_opt(),
                        StateName :: atom(),
                        StateData :: term(),
                        Timeout :: timeout().

Enter the gen_fsm receive loop.

With argument FsmName equivalent to enter_loop/6 with Timeout = infinity.

With argument Timeout equivalent to enter_loop/6 but the started server is not registered as for start_link/3.

enter_loop(Module, Options, StateName, StateData, FsmName, Timeout)

This function is deprecated. gen_fsm:enter_loop/6 is deprecated; use the 'gen_statem' module instead.

-spec enter_loop(Module, Options, StateName, StateData, FsmName, Timeout) -> no_return()
                    when
                        Module :: module(),
                        Options :: [enter_loop_opt()],
                        StateName :: atom(),
                        StateData :: term(),
                        FsmName :: fsm_name() | pid(),
                        Timeout :: timeout().

Enter the gen_fsm receive loop.

Makes an existing process into a gen_fsm process. Does not return, instead the calling process enters the gen_fsm receive loop and becomes a gen_fsm process. The process must have been started using one of the start functions in proc_lib. The user is responsible for any initialization of the process, including registering a name for it.

This function is useful when a more complex initialization procedure is needed than the gen_fsm behavior provides.

Module, Options, and FsmName have the same meanings as when calling start[_link]/3,4. However, the process must have been registered according to FsmName before this function is called.

StateName, StateData, and Timeout have the same meanings as in the return value of Module:init/1. The callback module Module does not need to export an init/1 function.

The function fails if the calling process was not started by a proc_lib start function, or if it is not registered according to FsmName.

reply(Caller, Reply)

This function is deprecated. gen_fsm:reply/2 is deprecated; use the 'gen_statem' module instead.

-spec reply(Caller, Reply) -> Result when Caller :: from(), Reply :: term(), Result :: term().

Send a reply to a caller.

This function can be used by a gen_fsm process to explicitly send a reply to a client process that called sync_send_event/2,3 or sync_send_all_state_event/2,3 when the reply cannot be defined in the return value of Module:StateName/3 or Module:handle_sync_event/4.

Caller must be the From argument provided to the callback function. Reply is any term given back to the client as the return value of sync_send_event/2,3 or sync_send_all_state_event/2,3.

Return value Result is not further defined, and is always to be ignored.

send_all_state_event(FsmRef, Event)

This function is deprecated. gen_fsm:send_all_state_event/2 is deprecated; use the 'gen_statem' module instead.

-spec send_all_state_event(FsmRef, Event) -> ok when FsmRef :: fsm_ref(), Event :: term().

Send an event asynchronously to a generic FSM.

Sends an event asynchronously to the FsmRef of the gen_fsm process and returns ok immediately. The gen_fsm process calls Module:handle_event/3 to handle the event.

For a description of the arguments, see send_event/2.

The difference between send_event/2 and send_all_state_event/2 is which callback function is used to handle the event. This function is useful when sending events that are handled the same way in every state, as only one handle_event clause is needed to handle the event instead of one clause in each state name function.

send_event(FsmRef, Event)

This function is deprecated. gen_fsm:send_event/2 is deprecated; use the 'gen_statem' module instead.

-spec send_event(FsmRef, Event) -> ok when FsmRef :: fsm_ref(), Event :: term().

Send an event asynchronously to a generic FSM.

Sends Event to the FsmRef of the gen_fsm process and returns ok immediately. The gen_fsm process calls Module:StateName/2 to handle the event, where StateName is the name of the current state of the gen_fsm process.

FsmRef can be any of the following:

The pid/0

Name, if the gen_fsm process is locally registered

{Name, Node}, if the gen_fsm process is locally registered at another node

{global, GlobalName}, if the gen_fsm process is globally registered

{via, Module, ViaName}, if the gen_fsm process is registered through an alternative process registry

Event is any term that is passed as one of the arguments to Module:StateName/2.

send_event_after(Time, Event)

This function is deprecated. gen_fsm:send_event_after/2 is deprecated; use the 'gen_statem' module instead.

-spec send_event_after(Time, Event) -> Ref
                          when Time :: non_neg_integer(), Event :: term(), Ref :: reference().

Send a delayed event internally in a generic FSM.

Sends a delayed event internally in the gen_fsm process that calls this function after Time milliseconds. Returns immediately a reference that can be used to cancel the delayed send using cancel_timer/1.

The gen_fsm process calls Module:StateName/2 to handle the event, where 'StateName' is the name of the current state of the gen_fsm process at the time the delayed event is delivered.

Event is any term that is passed as one of the arguments to Module:StateName/2.

start(Module, Args, Options)

This function is deprecated. gen_fsm:start/3 is deprecated; use the 'gen_statem' module instead.

-spec start(Module, Args, Options) -> Result
               when
                   Module :: module(),
                   Args :: term(),
                   Options :: [start_opt()],
                   Result :: {ok, Pid} | ignore | {error, Reason},
                   Pid :: pid(),
                   Reason :: term().

Create a standalone gen_fsm process, not registered.

Equivalent to start(Name, Mod, Args, Options) without registering a Name.

For a description of arguments and return values, see start_link/3,4.

start(FsmName, Module, Args, Options)

This function is deprecated. gen_fsm:start/4 is deprecated; use the 'gen_statem' module instead.

-spec start(FsmName, Module, Args, Options) -> Result
               when
                   FsmName :: fsm_name(),
                   Module :: module(),
                   Args :: term(),
                   Options :: [start_opt()],
                   Result :: {ok, Pid} | ignore | {error, Reason},
                   Pid :: pid(),
                   Reason :: {already_started, Pid} | term().

Create a standalone gen_fsm process.

The created process is not part of a supervision tree and thus has no supervisor.

For a description of arguments and return values, see start_link/3,4.

start_link(Module, Args, Options)

This function is deprecated. gen_fsm:start_link/3 is deprecated; use the 'gen_statem' module instead.

-spec start_link(Module, Args, Options) -> Result
                    when
                        Module :: module(),
                        Args :: term(),
                        Options :: [start_opt()],
                        Result :: {ok, Pid} | ignore | {error, Reason},
                        Pid :: pid(),
                        Reason :: term().

Create a gen_fsm process in a supervision tree, not registered.

Equivalent to start_link(Name, Mod, Args, Options) without registering a Name.

start_link(FsmName, Module, Args, Options)

This function is deprecated. gen_fsm:start_link/4 is deprecated; use the 'gen_statem' module instead.

-spec start_link(FsmName, Module, Args, Options) -> Result
                    when
                        FsmName :: fsm_name(),
                        Module :: module(),
                        Args :: term(),
                        Options :: [start_opt()],
                        Result :: {ok, Pid} | ignore | {error, Reason},
                        Pid :: pid(),
                        Reason :: {already_started, Pid} | term().

Create a gen_fsm process in a supervision tree.

The process is created as part of a supervision tree. The function is to be called, directly or indirectly, by the supervisor. For example, it ensures that the gen_fsm process is linked to the supervisor.

The gen_fsm process calls Module:init/1 to initialize. To ensure a synchronized startup procedure, start_link/3,4 does not return until Module:init/1 has returned.

If FsmName = {local, Name}, the gen_fsm process is registered locally as Name using register/2.

If FsmName = {global, GlobalName}, the gen_fsm process is registered globally as GlobalName using global:register_name/2.

If FsmName = {via, Module, ViaName}, the gen_fsm process registers with the registry represented by Module. The Module callback is to export the functions register_name/2, unregister_name/1, whereis_name/1, and send/2, which are to behave like the corresponding functions in global. Thus, {via, global, GlobalName} is a valid reference.

Module is the name of the callback module.

Args is any term that is passed as the argument to Module:init/1.

If option {timeout, Time} is present, the gen_fsm process is allowed to spend Time milliseconds initializing or it terminates and the start function returns {error, timeout}.

If option {debug, Dbgs} is present, the corresponding sys function is called for each item in Dbgs; see sys(3).

If option {spawn_opt, SOpts} is present, SOpts is passed as option list to the spawn_opt BIF that is used to spawn the gen_fsm process; see spawn_opt/2.

Note

Using spawn option monitor is not allowed, it causes the function to fail with reason badarg.

If the gen_fsm process is successfully created and initialized, the function returns {ok, Pid}, where Pid is the pid of the gen_fsm process. If a process with the specified FsmName exists already, the function returns {error, {already_started, Pid}}, where Pid is the pid of that process.

If Module:init/1 fails with Reason, the function returns {error, Reason}. If Module:init/1 returns {stop, Reason} or ignore, the process is terminated and the function returns {error, Reason} or ignore, respectively.

start_timer(Time, Msg)

This function is deprecated. gen_fsm:start_timer/2 is deprecated; use the 'gen_statem' module instead.

-spec start_timer(Time, Msg) -> Ref when Time :: non_neg_integer(), Msg :: term(), Ref :: reference().

Send a time-out event internally in a generic FSM.

Sends a time-out event internally in the gen_fsm process that calls this function after Time milliseconds. Returns immediately a reference that can be used to cancel the timer using cancel_timer/1.

The gen_fsm process calls Module:StateName/2 to handle the event, where 'StateName' is the name of the current state of the gen_fsm process at the time the time-out message is delivered.

Msg is any term that is passed in the time-out message, {timeout, Ref, Msg}, as one of the arguments to Module:StateName/2.

stop(FsmRef)

This function is deprecated. gen_fsm:stop/1 is deprecated; use the 'gen_statem' module instead.

-spec stop(FsmRef) -> ok when FsmRef :: fsm_ref().

Equivalent to stop(FsmRef, normal, infinity).

stop(FsmRef, Reason, Timeout)

This function is deprecated. gen_fsm:stop/3 is deprecated; use the 'gen_statem' module instead.

-spec stop(FsmRef, Reason, Timeout) -> ok
              when FsmRef :: fsm_ref(), Reason :: term(), Timeout :: timeout().

Synchronously stop a generic FSM.

Orders a generic finite state machine to exit with the specified Reason and waits for it to terminate. The gen_fsm process calls Module:terminate/3 before exiting.

The function returns ok if the generic finite state machine terminates with the expected reason. Any other reason than normal, shutdown, or {shutdown, Term} causes an error report to be issued using error_logger:format/2.

Timeout is an integer greater than zero that specifies how many milliseconds to wait for the generic FSM to terminate, or the atom infinity to wait indefinitely. If the generic finite state machine has not terminated within the specified time, a timeout exception is raised.

If the process does not exist, a noproc exception is raised.

sync_send_all_state_event(FsmRef, Event)

This function is deprecated. gen_fsm:sync_send_all_state_event/2 is deprecated; use the 'gen_statem' module instead.

-spec sync_send_all_state_event(FsmRef, Event) -> Reply
                                   when FsmRef :: fsm_ref(), Event :: term(), Reply :: term().

Equivalent to sync_send_all_state_event(FsmRef, Event, 5000).

sync_send_all_state_event(FsmRef, Event, Timeout)

This function is deprecated. gen_fsm:sync_send_all_state_event/3 is deprecated; use the 'gen_statem' module instead.

-spec sync_send_all_state_event(FsmRef, Event, Timeout) -> Reply
                                   when
                                       FsmRef :: fsm_ref(),
                                       Event :: term(),
                                       Timeout :: timeout(),
                                       Reply :: term().

Send an event synchronously to a generic FSM.

Sends an event to the FsmRef of the gen_fsm process and waits until a reply arrives or a time-out occurs. The gen_fsm process calls Module:handle_sync_event/4 to handle the event.

For a description of FsmRef and Event, see send_event/2. For a description of Timeout and Reply, see sync_send_event/3.

For a discussion about the difference between sync_send_event and sync_send_all_state_event, see send_all_state_event/2.

sync_send_event(FsmRef, Event)

This function is deprecated. gen_fsm:sync_send_event/2 is deprecated; use the 'gen_statem' module instead.

-spec sync_send_event(FsmRef, Event) -> Reply when FsmRef :: fsm_ref(), Event :: term(), Reply :: term().

Equivalent to sync_send_event(FsmRef, Event, 5000).

sync_send_event(FsmRef, Event, Timeout)

This function is deprecated. gen_fsm:sync_send_event/3 is deprecated; use the 'gen_statem' module instead.

-spec sync_send_event(FsmRef, Event, Timeout) -> Reply
                         when
                             FsmRef :: fsm_ref(), Event :: term(), Timeout :: timeout(), Reply :: term().

Send an event synchronously to a generic FSM.

Sends an event to the FsmRef of the gen_fsm process and waits until a reply arrives or a time-out occurs. The gen_fsm process calls Module:StateName/3 to handle the event, where 'StateName' is the name of the current state of the gen_fsm process.

For a description of FsmRef and Event, see send_event/2.

Timeout is an integer greater than zero that specifies how many milliseconds to wait for a reply, or the atom infinity to wait indefinitely. If no reply is received within the specified time, the function call fails.

Return value Reply is defined in the return value of Module:StateName/3

Note

The ancient behavior of sometimes consuming the server exit message if the server died during the call while linked to the client was removed in Erlang 5.6/OTP R12B.

gen_server behaviour

Generic server behavior.

This behavior module provides the server in a client-server relation. A generic server process (gen_server) implemented using this module has a standard set of interface functions and includes functionality for tracing and error reporting. It also fits into an OTP supervision tree. For more information, see section gen_server Behaviour in OTP Design Principles.

A gen_server process assumes all specific parts to be located in a callback module exporting a predefined set of functions. The relationship between the behavior functions and the callback functions is as follows:

gen_server module            Callback module
-----------------            ---------------
gen_server:start
gen_server:start_monitor
gen_server:start_link -----> Module:init/1

gen_server:stop       -----> Module:terminate/2

gen_server:call
gen_server:send_request
gen_server:multi_call -----> Module:handle_call/3

gen_server:cast
gen_server:abcast     -----> Module:handle_cast/2

-                     -----> Module:handle_info/2

-                     -----> Module:handle_continue/2

-                     -----> Module:terminate/2

-                     -----> Module:code_change/3

If a callback function fails or returns a bad value, the gen_server process terminates. However, an exception of class throw is not regarded as an error but as a valid return, from all callback functions.

A gen_server process handles system messages as described in sys. The sys module can be used for debugging a gen_server process.

Notice that a gen_server process does not trap exit signals automatically, this must be explicitly initiated in the callback module.

Unless otherwise stated, all functions in this module fail if the specified gen_server process does not exist or if bad arguments are specified.

The gen_server process can go into hibernation (see erlang:hibernate/3) if a callback function specifies 'hibernate' instead of a time-out value. This can be useful if the server is expected to be idle for a long time. However, use this feature with care, as hibernation implies at least two garbage collections (when hibernating and shortly after waking up) and is not something you want to do between each call to a busy server.

If the gen_server process needs to perform an action after initialization or to break the execution of a callback into multiple steps, it can return {continue, Continue} in place of the time-out or hibernation value, which will invoke the Module:handle_continue/2 callback, before receiving any external message / request.

If the gen_server process terminates, e.g. as a result of a function in the callback module returning {stop,Reason,NewState}, an exit signal with this Reason is sent to linked processes and ports. See Processes in the Reference Manual for details regarding error handling using exit signals.

Note

For some important information about distributed signals, see the Blocking Signaling Over Distribution section in the Processes chapter of the Erlang Reference Manual. Blocking signaling can, for example, cause call time-outs in gen_server to be significantly delayed.

See Also

gen_event, gen_statem, proc_lib, supervisor, sys

Summary

Types

action()

Callback and server loop action.

enter_loop_opt()

Server start options for the start or enter_loop functions.

format_status()

A map that describes the gen_server status.

from()

A call's reply destination.

reply_tag()

A handle that associates a reply to the corresponding request.

request_id()

An opaque request identifier. See send_request/2 for details.

request_id_collection()

An opaque collection of request identifiers (request_id/0).

response_timeout()

Response time-out for an asynchronous call.

server_name()

Server name specification: local, global, or via registered.

server_ref()

Server specification: pid/0 or registered server_name/0.

start_mon_ret()

Return value from the start_monitor/3,4 functions.

start_opt()

Server start options for the start functions.

start_ret()

Return value from the start/3,4 and start_link/3,4 functions.

timeout_option()

Time-out timer start option, to select absolute time of expiry.

Callbacks

code_change(OldVsn, State, Extra)

Update the server state after code change.

format_status(Status)

Format/limit the status value.

format_status(Opt, StatusData) deprecated

Format/limit the status value.

handle_call(Request, From, State)

Handle a call.

handle_cast(Request, State)

Handle a cast message.

handle_continue(Info, State)

Handle a callback continuation.

handle_info(Info, State)

Handle an info message (regular process message).

init(Args)

Initialize the server.

terminate(Reason, State)

Handle server termination.

Functions

abcast(Name, Request)

Cast a request to multiple nodes.

abcast(Nodes, Name, Request)

Cast a request to multiple nodes.

call(ServerRef, Request)

Equivalent to call(ServerRef, Request, 5000).

call(ServerRef, Request, Timeout)

Call a server: send request and wait for response.

cast(ServerRef, Request)

Cast a request to a server.

check_response(Msg, ReqId)

Check if a received message is a request response.

check_response(Msg, ReqIdCollection, Delete)

Check if a received message is a request response in a collection.

enter_loop(Module, Options, State)

Equivalent to enter_loop(Mod, Options, State, self()).

enter_loop/4

Make the calling process become a gen_server process.

enter_loop(Module, Options, State, ServerName, Action)

Make the calling process become a gen_server process.

multi_call(Name, Request)

Call servers on multiple nodes in parallel.

multi_call(Nodes, Name, Request)

Equivalent to multi_call(Nodes, Name, Request, infinity).

multi_call(Nodes, Name, Request, Timeout)

Call servers on multiple nodes in parallel.

receive_response(ReqId, Timeout)

Receive a request response.

receive_response(ReqIdCollection, Timeout, Delete)

Receive a request response in a collection.

reply(Client, Reply)

Send a reply to a client.

reqids_add(ReqId, Label, ReqIdCollection)

Store a request identifier in a colletion.

reqids_new()

Create an empty request identifier collection.

reqids_size(ReqIdCollection)

Returns the number of request identifiers in ReqIdCollection.

reqids_to_list(ReqIdCollection)

Convert a request identifier collection to a list.

send_request(ServerRef, Request)

Send an asynchronous call request.

send_request(ServerRef, Request, Label, ReqIdCollection)

Send an asynchronous call request and add it to a request identifier collection.

start(Module, Args, Options)

Start a server, neither linked nor registered.

start(ServerName, Module, Args, Options)

Start a server, registered but not linked.

start_link(Module, Args, Options)

Start a server, linked but not registered.

start_link(ServerName, Module, Args, Options)

Start a server, linked and registered.

start_monitor(Module, Args, Options)

Start a server, monitored but neither linked nor registered.

start_monitor(ServerName, Module, Args, Options)

Start a server, monitored and registered, but not linked.

stop(ServerRef)

Equivalent to stop(ServerRef, normal, infinity).

stop(ServerRef, Reason, Timeout)

Stop a server.

wait_response(ReqId, WaitTime)

Wait for a request response.

wait_response(ReqIdCollection, WaitTime, Delete)

Wait for any request response in a collection.

Types

action()

-type action() ::
          (Time :: timeout()) |
          hibernate |
          {timeout, Time :: timeout(), Message :: term()} |
          {timeout,
           Time :: timeout(),
           Message :: term(),
           Options :: timeout_option() | [timeout_option()]} |
          {hibernate, Time :: timeout(), Message :: term()} |
          {hibernate,
           Time :: timeout(),
           Message :: term(),
           Options :: timeout_option() | [timeout_option()]} |
          {continue, Continue :: term()}.

Callback and server loop action.

Returned by the callbacks Module:init/1, Module:handle_call/3, Module:handle_cast/2, Module:handle_info/2, and Module:handle_continue/2. It is also the last argument to enter_loop/4,5.

action/0 is one of:

Time ::timeout() - If the action is an integer Time, a time-out occurs unless a request or a message is received within that many milliseconds. A time-out is represented by the atom timeout as the Info argument to be handled by the Module:handle_info/2 callback function. The action infinity can be used to wait indefinitely, which is the default when there is no action/0 specified.

For Time = 0, if there is a request or message waiting to be received, it interrupts (cancels) the time-out.

Note

A system message restarts the time-out, which is a known and unfortunate flaw in its implementation. This also applies to stray (cancelled) timer messages from the {timeout|hibernate, ...} time-outs described below, so it is recommended to not use them in combination with this legacy time-out type.

{timeout, Time, Message}
{timeout, Time, Message, Options} - Like Time above, but the delivered Info argument is specified by Message, and the time-out is not affected by system messages.

Options can be used to trigger the time-out at an absolute point in time instead. See timeout_option/0 and erlang:start_timer/4 for details.

A relative time-out with Time = 0 is immediately delivered, before any request or message is received, including system messages.

A time-out with Time = infinity will never be delivered so the action is ignored; no time-out is started.

{hibernate, Time, Message}
{hibernate, Time, Message, Options} - A combination of the action hibernate below, and {timeout, Time, Message} or {timeout, Time, Message, Options} above.

The process goes into hibernation while waiting for the next request or message to arrive, or for the time-out to expire.

hibernate - The process goes into hibernation (by calling erlang:hibernate/0), waiting for the next request or message to arrive

{continue, Continue} - The process will immediately execute the Module:handle_continue/2 callback function, with Continue as the first argument.

enter_loop_opt()

-type enter_loop_opt() ::
          {hibernate_after, HibernateAfterTimeout :: timeout()} | {debug, Dbgs :: [sys:debug_option()]}.

Server start options for the start or enter_loop functions.

Options that can be used when starting a gen_server server through enter_loop/3-5 or the start functions such as start_link/3,4.

{hibernate_after, HibernateAfterTimeout} - Specifies that the gen_server process awaits any message for HibernateAfterTimeout milliseconds and if no message is received, the process goes into hibernation automatically (by calling proc_lib:hibernate/3).

{debug, Dbgs} - For every entry in Dbgs, the corresponding function in sys is called.

format_status()

-type format_status() ::
          #{state => term(), message => term(), reason => term(), log => [sys:system_event()]}.

A map that describes the gen_server status.

The keys are:

state - The internal state of the gen_server process.

message - The message that caused the server to terminate.

reason - The reason that caused the server to terminate.

log - The sys log of the server.

New associations may be added to the status map without prior notice.

from()

-type from() :: {Client :: pid(), Tag :: reply_tag()}.

A call's reply destination.

Destination, given to the gen_server as the first argument to the callback function Module:handle_call/3, to be used by the when replying through reply/2 (instead of through the callback function's return value), to the process Client that has called the gen_server using call/2,3. Tag is a term that is unique for this call/request instance.

reply_tag()

-opaque reply_tag()

A handle that associates a reply to the corresponding request.

request_id()

-opaque request_id()

An opaque request identifier. See send_request/2 for details.

request_id_collection()

-opaque request_id_collection()

An opaque collection of request identifiers (request_id/0).

Each request identifier can be associated with a label chosen by the user. For more information see reqids_new/0.

response_timeout()

(not exported)

-type response_timeout() :: timeout() | {abs, integer()}.

Response time-out for an asynchronous call.

Used to set a time limit on how long to wait for a response using either receive_response/2, receive_response/3, wait_response/2, or wait_response/3. The time unit used is millisecond.

Currently valid values:

0..4294967295 - Time-out relative to current time in milliseconds.

infinity - Infinite time-out. That is, the operation will never time out.

{abs, Timeout} - An absolute Erlang monotonic time time-out in milliseconds. That is, the operation will time out when erlang:monotonic_time(millisecond) returns a value larger than or equal to Timeout. Timeout is not allowed to identify a time further into the future than 4294967295 milliseconds. Specifying the time-out using an absolute value is especially handy when you have a deadline for responses corresponding to a complete collection of requests (request_id_collection/0), since you do not have to recalculate the relative time until the deadline over and over again.

server_name()

-type server_name() ::
          {local, LocalName :: atom()} |
          {global, GlobalName :: term()} |
          {via, RegMod :: module(), ViaName :: term()}.

Server name specification: local, global, or via registered.

To be used when starting a gen_server. See functions start/3,4, start_link/3,4, start_monitor/3,4, enter_loop/3,4,5, and the type server_ref/0.

{local, LocalName} - Register the gen_server locally as LocalName using register/2.

{global, GlobalName} - Register the gen_server process id globally as GlobalName using global:register_name/2.

{via, RegMod, ViaName} - Register the gen_server process with the registry represented by RegMod. The RegMod callback is to export the functions register_name/2, unregister_name/1, whereis_name/1, and send/2, which are to behave like the corresponding functions in global. Thus, {via, global, GlobalName} is a valid reference equivalent to {global, GlobalName}.

server_ref()

-type server_ref() ::
          pid() |
          (LocalName :: atom()) |
          {Name :: atom(), Node :: atom()} |
          {global, GlobalName :: term()} |
          {via, RegMod :: module(), ViaName :: term()}.

Server specification: pid/0 or registered server_name/0.

To be used when addressing a gen_server. See call/2,3, cast/2, send_request/2, check_response/2, wait_response/2, stop/2,3 and the type server_name/0.

It can be:

pid/0 - The gen_server's process identifier.

LocalName - The gen_server is locally registered as LocalName with register/2.

{Name,Node} - The gen_server is locally registered on another node.

{global, GlobalName} - The gen_server is globally registered in global.

{via, RegMod, ViaName} - The gen_server is registered in an alternative process registry. See the same term described for server_name/0.

start_mon_ret()

-type start_mon_ret() ::
          {ok, {Pid :: pid(), MonRef :: reference()}} | ignore | {error, Reason :: term()}.

Return value from the start_monitor/3,4 functions.

The same as type start_ret/0 except that for a succesful start it returns both the process identifier Pid and a monitor/2,3 MonRef.

start_opt()

-type start_opt() ::
          {timeout, Timeout :: timeout()} |
          {spawn_opt, SpawnOptions :: [proc_lib:start_spawn_option()]} |
          enter_loop_opt().

Server start options for the start functions.

Options that can be used when starting a gen_server server through, for example, start_link/3,4.

{timeout, Timeout} - How many milliseconds the gen_server process is allowed to spend initializing or it is terminated and the start function returns {error, timeout}.

{spawn_opt, SpawnOptions} - The SpawnOptions option list is passed to the function used to spawn the gen_server; see proc_lib:start_spawn_option/0).

Note

Using spawn option monitor is not allowed - it causes a badarg failure.

enter_loop_opt/0 - See the type enter_loop_opt/0 below for more start options that are also allowed by enter_loop/3,4,5.

start_ret()

-type start_ret() :: {ok, Pid :: pid()} | ignore | {error, Reason :: term()}.

Return value from the start/3,4 and start_link/3,4 functions.

{ok, Pid} - The gen_server process was succesfully created and initialized, with the process identifier Pid.

{error, {already_started, OtherPid}} - A process with the specified ServerName exists already with the process identifier OtherPid. This function failed to start a gen_server. It exited with reason normal before calling Module:init/1.

{error, timeout} - The gen_server process failed to initialize since Module:init/1 did not return within the start time-out. The gen_server process was killed with exit(_, kill).

ignore - The gen_server process failed to initialize since Module:init/1 returned ignore.

{error,Reason} - The gen_server process failed to initialize since Module:init/1 returned {stop,Reason}, {error,Reason}, or it failed with reason Reason.

See Module:init/1 about the exit reason for the gen_server process when it fails to initialize.

timeout_option()

(not exported)

-type timeout_option() :: {abs, Abs :: boolean()}.

Time-out timer start option, to select absolute time of expiry.

If Abs is true an absolute timer is started, and if it is false a relative, which is the default. See erlang:start_timer/4 for details.

Callbacks

code_change(OldVsn, State, Extra)

(optional)

-callback code_change(OldVsn :: term() | {down, term()}, State :: term(), Extra :: term()) ->
                         {ok, NewState :: term()} | {error, Reason :: term()}.

Update the server state after code change.

This function is called by a gen_server process when it is to update its internal state during a release upgrade/downgrade, that is, when the instruction {update, Module, Change, ...}, is specified in the appup file.

For more information, see section Release Handling Instructions in OTP Design Principles.

For an upgrade, OldVsn is Vsn, and for a downgrade, OldVsn is {down,Vsn}. Vsn is defined by the vsn attribute(s) of the old version of the callback module Module. If no such attribute is defined, the version is the checksum of the Beam file.

State is the internal state of the gen_server process.

Extra is passed "as is" from the {advanced,Extra} part of the update instruction.

If successful, the function must return the updated internal state.

If the function returns {error,Reason}, the ongoing upgrade fails and rolls back to the old release.

Note

If a release upgrade/downgrade with Change = {advanced, Extra} specified in the .appup file is made when Module:code_change/3 is not implemented, the callback call will crash with an undef error reason.

format_status(Status)

(since OTP 25.0) (optional)

-callback format_status(Status) -> NewStatus when Status :: format_status(), NewStatus :: format_status().

Format/limit the status value.

This function is called by a gen_server process in in order to format/limit the server state for debugging and logging purposes.

It is called in the following situations:

sys:get_status/1,2 is invoked to get the gen_server status.

The gen_server process terminates abnormally and logs an error.

This callback is used to limit the status of the process returned by sys:get_status/1,2 or sent to logger.

The callback gets a map Status describing the current status and shall return a map NewStatus with the same keys, but it may transform some values.

Two possible use cases for this callback is to remove sensitive information from the state to prevent it from being printed in log files, or to compact large irrelevant status items that would only clutter the logs.

Example:

format_status(Status) ->
  maps:map(
    fun(state,State) ->
            maps:remove(private_key, State);
       (message,{password, _Pass}) ->
            {password, removed};
       (_,Value) ->
            Value
    end, Status).

Note

This callback is optional, so callback modules need not export it. The gen_server module provides a default implementation of this function that returns the callback module state.

If this callback is exported but fails, to hide possibly sensitive data, the default function will instead return the fact that Module:format_status/1 has crashed.

format_status(Opt, StatusData)

(since OTP R13B04) (optional)

This callback is deprecated. the callback gen_server:format_status(_,_) is deprecated; use format_status/1 instead.

-callback format_status(Opt, StatusData) -> Status
                           when
                               Opt :: normal | terminate,
                               StatusData :: [PDict | State],
                               PDict :: [{Key :: term(), Value :: term()}],
                               State :: term(),
                               Status :: term().

Format/limit the status value.

This function is called by a gen_server process in in order to format/limit the server state for debugging and logging purposes.

It is called in the following situations:

One of sys:get_status/1,2 is invoked to get the gen_server status. Opt is set to the atom normal.

The gen_server process terminates abnormally and logs an error. Opt is set to the atom terminate.

This function is useful for changing the form and appearance of the gen_server status for these cases. A callback module wishing to change the sys:get_status/1,2 return value, as well as how its status appears in termination error logs, exports an instance of Module:format_status/2 that returns a term describing the current status of the gen_server process.

PDict is the current value of the process dictionary of the gen_server process..

State is the internal state of the gen_server process.

The function is to return Status, a term that changes the details of the current state and status of the gen_server process. There are no restrictions of the form Status can take, but for the sys:get_status/1,2 case (when Opt is normal), the recommended form for the Status value is [{data, [{"State", Term}]}], where Term provides relevant details of the gen_server state. Following this recommendation is not required, but it makes the callback module status consistent with the rest of the sys:get_status/1,2 return value.

One use for this function is to return compact alternative state representations to avoid that large state terms are printed in log files.

Note

This callback is optional, so callback modules need not export it. The gen_server module provides a default implementation of this function that returns the callback module state.

handle_call(Request, From, State)

-callback handle_call(Request :: term(), From :: from(), State :: term()) ->
                         {reply, Reply :: term(), NewState :: term()} |
                         {reply, Reply :: term(), NewState :: term(), Action :: action()} |
                         {noreply, NewState :: term()} |
                         {noreply, NewState :: term(), Action :: action()} |
                         {stop, Reason :: term(), Reply :: term(), NewState :: term()} |
                         {stop, Reason :: term(), NewState :: term()}.

Handle a call.

Whenever a gen_server process receives a request sent using call/2,3, multi_call/2,3,4, or send_request/2,4, this function is called to handle the request.

State is the internal state of the gen_server process, and NewState a possibly updated one.

Request is passed from the same argument provided to call or multi_call.

The return value Result is interpreted as follows:

{reply,Reply,NewState}
{reply,Reply,NewState,_} - The Reply value is sent back to the client request and there becomes its return value.

The gen_server process continues executing with the possibly updated internal state NewState.

{noreply,NewState}
{noreply,NewState,_} - The gen_server process continues executing with the possibly updated internal state NewState.

A reply to the client request has to be created by calling reply(From, Reply), either in this or in a later callback.

{reply,_,_,Action}
{noreply,_,Action} - Action is described by the action/0 type.

{stop,Reason,NewState}
{stop,Reason,Reply,NewState} - The gen_server process will call Module:terminate(Reason,NewState), and then terminate.

{stop,_,Reply,_} will create a reply to the client request just as {reply,Reply,...} while {stop,_,_} will not, so just as for {noreply,NewState,...} a reply has to be created by calling reply(From, Reply) before returning {stop,_,_}.

handle_cast(Request, State)

-callback handle_cast(Request :: term(), State :: term()) ->
                         {noreply, NewState :: term()} |
                         {noreply, NewState :: term(), Action :: action()} |
                         {stop, Reason :: term(), NewState :: term()}.

Handle a cast message.

Whenever a gen_server process receives a request sent using cast/2 or abcast/2,3, this function is called to handle the request.

For a description of the arguments and possible return values, see Module:handle_call/3.

handle_continue(Info, State)

(since OTP 21.0) (optional)

-callback handle_continue(Info :: term(), State :: term()) ->
                             {noreply, NewState :: term()} |
                             {noreply, NewState :: term(), Action :: action()} |
                             {stop, Reason :: term(), NewState :: term()}.

Handle a callback continuation.

This function is called by a gen_server process whenever a previous callback returns one of the tuples containing {continue, Continue}. The call is invoked immediately after the previous callback, which makes it useful for performing work after initialization or, for splitting the work in a callback into multiple steps, updating the process state along the way.

For a description of the other arguments and possible return values, see Module:handle_call/3.

Note

This callback is optional, so callback modules need to export it only if they return one of the tuples containing {continue,Continue} from another callback. If such a {continue,_} tuple is used and the callback is not implemented, the process will exit with undef error.

handle_info(Info, State)

(optional)

-callback handle_info(Info :: timeout | term(), State :: term()) ->
                         {noreply, NewState :: term()} |
                         {noreply, NewState :: term(), Action :: action()} |
                         {stop, Reason :: term(), NewState :: term()}.

Handle an info message (regular process message).

This function is called by a gen_server process when a time-out occurs or when it receives any other message than a synchronous or asynchronous request (or a system message).

Info is either the atom timeout, if a time-out has occurred, or the received message.

For a description of the other arguments and possible return values, see Module:handle_call/3.

Note

This callback is optional, so callback modules need not export it. The gen_server module provides a default implementation of this function that logs about the unexpected Info message, drops it and returns {noreply, State}.

init(Args)

-callback init(Args :: term()) ->
                  {ok, State :: term()} |
                  {ok, State :: term(), Action :: action()} |
                  {stop, Reason :: term()} |
                  ignore |
                  {error, Reason :: term()}.

Initialize the server.

Whenever a gen_server process is started using start/3,4, start_monitor/3,4, or start_link/3,4, this function is called by the new process to initialize the server.

Args is the Args argument provided to the start function.

The return value Result is interpreted as follows:

{ok,State}
{ok,State,_} - Initialization was succesful and State is the internal state of the gen_server process.

{ok,_,Action} - Action is described by the action/0 type.

{stop,Reason} - Initialization failed. The gen_server process exits with reason Reason.

{error,Reason} since OTP 26.0
ignore - Initialization failed. The gen_server process exits with reason normal.

See function start_link/3,4's return value start_ret/0 in these different cases.

terminate(Reason, State)

(optional)

-callback terminate(Reason :: normal | shutdown | {shutdown, term()} | term(), State :: term()) -> term().

Handle server termination.

This function is called by a gen_server process when it is about to terminate.

It is to be the opposite of Module:init/1 and do any necessary cleaning up. When it returns, the gen_server process terminates with Reason. The return value is ignored.

Reason is a term denoting the stop reason and State is the internal state of the gen_server process.

Reason depends on why the gen_server process is terminating. If it is because another callback function has returned a stop tuple {stop,..}, Reason has the value specified in that tuple. If it is because of a failure, Reason is the error reason.

If the gen_server process is part of a supervision tree and is ordered by its supervisor to terminate, this function is called with Reason=shutdown if the following conditions apply:

The gen_server process has been set to trap exit signals.

The shutdown strategy as defined in the child specification of the supervisor is an integer time-out value, not brutal_kill.

Even if the gen_server process is not part of a supervision tree, this function is called if it receives an 'EXIT' message from its parent. Reason is the same as in the 'EXIT' message.

If the gen_server process does not trap exits, the gen_server process terminates immediately.

Notice that for any other reason than normal, shutdown, or {shutdown,Term}, see stop/3, the gen_server process is assumed to terminate because of an error, and an error report is issued using logger.

When the gen_server process exits, an exit signal with the same reason is sent to linked processes and ports.

Note

This callback is optional, so callback modules need not export it. The gen_server module provides a default implementation with no cleanup.

Functions

abcast(Name, Request)

-spec abcast(Name :: atom(), Request :: term()) -> abcast.

Cast a request to multiple nodes.

Equivalent to abcast(Nodes, Name, Request) where Nodes is all nodes connected to the calling node, including the calling node itself.

abcast(Nodes, Name, Request)

-spec abcast(Nodes :: [node()], Name :: atom(), Request :: term()) -> abcast.

Cast a request to multiple nodes.

Sends an asynchronous request to the gen_server processes locally registered as Name at the specified nodes. The function returns immediately and ignores nodes that do not exist, or where the gen_server Name does not exist. The gen_server processes call Module:handle_cast/2 to handle the request.

For a description of the arguments, see multi_call/2,3,4.

call(ServerRef, Request)

-spec call(ServerRef :: server_ref(), Request :: term()) -> Reply :: term().

Equivalent to call(ServerRef, Request, 5000).

call(ServerRef, Request, Timeout)

-spec call(ServerRef :: server_ref(), Request :: term(), Timeout :: timeout()) -> Reply :: term().

Call a server: send request and wait for response.

Makes a synchronous call to the ServerRef of the gen_server process by sending a request and waiting until a reply arrives or a time-out occurs. The gen_server process calls Module:handle_call/3 to handle the request.

See also ServerRef's type server_ref/0.

Request is any term that is passed as the first argument to Module:handle_call/3.

Timeout is an integer that specifies how many milliseconds to wait for a reply, or the atom infinity to wait indefinitely. If no reply is received within the specified time, this function exits the calling process with an exit term containing Reason = timeout as described below.

Note

Before OTP 24, if the caller uses (try...)catch to avoid process exit, and the server happens to just be late with the reply, it may arrive to the process message queue any time later. The calling process must therefore after catching a time-out exit be prepared to receive garbage message(s) of the form {reference(), _} and deal with them appropriately (discard them) so they do not clog the process message queue, or gets mistaken for other messages.

Starting with OTP 24, gen_server:call uses process aliases, so late replies will not be received.

The return value Reply is passed from the return value of Module:handle_call/3.

This call may exit the calling process with an exit term on the form {Reason, Location} where Location = {gen_server, call, ArgList} and Reason can be (at least) one of:

timeout - The call was aborted after waiting Timeout milliseconds for a reply, as described above.

noproc - The ServerRef refers to a server by name (it is not a pid/0) and looking up the server process failed, or the pid/0 was already terminated.

{nodedown,Node} - The ServerRef refers to a server on the remote node Node and the connection to that node failed.

calling_self - A call to self/0 would hang indefinitely.

shutdown - The server was stopped during the call by its supervisor. See also stop/3.

normal
{shutdown,Term} - The server stopped during the call by returning {stop,Reason,_} from one of its callbacks without replying to this call. See also stop/3.

_OtherTerm - The server process exited during the call, with reason Reason. Either by returning {stop,Reason,_} from one of its callbacks (without replying to this call), by raising an exception, or due to getting an exit signal it did not trap.

cast(ServerRef, Request)

-spec cast(ServerRef :: server_ref(), Request :: term()) -> ok.

Cast a request to a server.

Sends an asynchronous request to the gen_server ServerRef and returns ok immediately, ignoring if the destination node or gen_server process does not exist.

The gen_server process calls Module:handle_cast(Request, _) to handle the request.

check_response(Msg, ReqId)

(since OTP 23.0)

-spec check_response(Msg, ReqId) -> Result
                        when
                            Msg :: term(),
                            ReqId :: request_id(),
                            Response ::
                                {reply, Reply :: term()} | {error, {Reason :: term(), server_ref()}},
                            Result :: Response | no_reply.

Check if a received message is a request response.

Checks if Msg is a response corresponding to the request identifier ReqId. The request must have been made by send_request/2, and by the same process calling this function.

If Msg is a reply to the handle ReqId the result of the request is returned in Reply. Otherwise this function returns no_reply and no cleanup is done, and thus the function shall be invoked repeatedly until the response is returned.

The return value Reply is passed from the return value of Module:handle_call/3.

If the gen_statem server process has died when this function is called, that is; Msg reports the server's death, this function returns an error return with the exit Reason.

check_response(Msg, ReqIdCollection, Delete)

(since OTP 25.0)

-spec check_response(Msg, ReqIdCollection, Delete) -> Result
                        when
                            Msg :: term(),
                            ReqIdCollection :: request_id_collection(),
                            Delete :: boolean(),
                            Response ::
                                {reply, Reply :: term()} | {error, {Reason :: term(), server_ref()}},
                            Result ::
                                {Response,
                                 Label :: term(),
                                 NewReqIdCollection :: request_id_collection()} |
                                no_request | no_reply.

Check if a received message is a request response in a collection.

Check if Msg is a response corresponding to a request identifier stored in ReqIdCollection. All request identifiers of ReqIdCollection must correspond to requests that have been made using send_request/2 or send_request/4, by the process calling this function.

The Label in the response equals the Label associated with the request identifier that the response corresponds to. The Label of a request identifier is associated when storing the request id in a collection, or when sending the request using send_request/4.

Compared to check_response/2, the returned result or exception associated with a specific request identifier will be wrapped in a 3-tuple {Response, Label, NewReqIdCollection}. Response is the value that would have been produced by check_response/2, Label is the value associated with the specific request identifier and NewReqIdCollection is a possibly modified request identifier collection.

If ReqIdCollection is empty, no_request will be returned.

If Msg does not correspond to any of the request identifiers in ReqIdCollection, no_reply is returned.

If Delete is true, the association with Label has been deleted from ReqIdCollection in the resulting NewReqIdCollection. If Delete is false, NewReqIdCollection will equal ReqIdCollection. Note that deleting an association is not for free and that a collection containing already handled requests can still be used by subsequent calls to check_response/3, receive_response/3, and wait_response/3.

However, without deleting handled associations, the above calls will not be able to detect when there are no more outstanding requests to handle, so you will have to keep track of this some other way than relying on a no_request return. Note that if you pass a collection only containing associations of already handled or abandoned requests to this function, it will always return no_reply.

enter_loop(Module, Options, State)

-spec enter_loop(Module :: module(), Options :: [enter_loop_opt()], State :: term()) -> no_return().

Equivalent to enter_loop(Mod, Options, State, self()).

enter_loop/4

-spec enter_loop(Module :: module(),
                 Options :: [enter_loop_opt()],
                 State :: term(),
                 ServerName :: server_name() | pid()) ->
                    no_return();
                (Module :: module(), Options :: [enter_loop_opt()], State :: term(), Action :: action()) ->
                    no_return().

Make the calling process become a gen_server process.

With argument ServerName equivalent to enter_loop(Module, Options, State, ServerName, infinity).

With argument Action equivalent to enter_loop(Module, Options, State, self(), Action).

enter_loop(Module, Options, State, ServerName, Action)

-spec enter_loop(Module :: module(),
                 Options :: [enter_loop_opt()],
                 State :: term(),
                 ServerName :: server_name() | pid(),
                 Action :: action()) ->
                    no_return().

Make the calling process become a gen_server process.

Does not return, instead the calling process enters the gen_server process receive loop and becomes a gen_server process. The process must have been started using one of the start functions in proc_lib. The user is responsible for any initialization of the process, including registering a name for it.

This function is useful when a more complex initialization procedure is needed than the gen_server Module:init/1; callback provides.

Module, Options, and ServerName have the same meanings as when calling start[_link|_monitor]/3,4 or ServerName can be self/0 for an anonymous server, which is the same as calling an enter_loop/3,4 function without a ServerName argument. However, if ServerName is specified (and not as self/0), the process must have been registered accordingly before this function is called.

State has the same meanings as in the return value of Module:init/1, which is not called when enter_loop/3,4,5 is used. Note that to adhere to the gen_server Behaviour such a callback function needs to be defined, and it might as well be the one used when starting the gen_server process through proc_lib, and then be the one that calls enter_loop/3,4,5. But if such a Module:init/1 function, in for example error cases, cannot call enter_loop/3,4,5, it should return a value that follows the type specification for Module:init/1 such as ignore, although that value will be lost when returning to the spawning function.

Action is described by the action/0 type.

This function fails if the calling process was not started by a proc_lib start function, or if it is not registered according to ServerName.

multi_call(Name, Request)

-spec multi_call(Name :: atom(), Request :: term()) ->
                    {Replies :: [{Node :: node(), Reply :: term()}], BadNodes :: [node()]}.

Call servers on multiple nodes in parallel.

Equivalent to multi_call(Nodes, Name, Request) where Nodes is all nodes connected to the calling node, including the calling node itself.

multi_call(Nodes, Name, Request)

-spec multi_call(Nodes :: [node()], Name :: atom(), Request :: term()) ->
                    {Replies :: [{Node :: node(), Reply :: term()}], BadNodes :: [node()]}.

Equivalent to multi_call(Nodes, Name, Request, infinity).

multi_call(Nodes, Name, Request, Timeout)

-spec multi_call(Nodes :: [node()], Name :: atom(), Request :: term(), Timeout :: timeout()) ->
                    {Replies :: [{Node :: node(), Reply :: term()}], BadNodes :: [node()]}.

Call servers on multiple nodes in parallel.

Makes a synchronous call to all gen_server processes locally registered as Name at the specified nodes, by first sending the request to the nodes, and then waiting for the replies. The gen_server processes on the nodes call Module:handle_call/3 to handle the request.

The function returns a tuple {Replies, BadNodes}, where Replies is a list of {Node, Reply} tuples, and BadNodes is a list of nodes that either did not exist, where Name was not a registered gen_server, or where it did not reply.

Nodes is a list of node names to which the request is to be sent.

Name is the locally registered name for each gen_server process.

Request is any term that is passed as the first argument to Module:handle_call/3.

Timeout is an integer that specifies how many milliseconds to wait for all replies, or the atom infinity to wait indefinitely. If no reply is received from a node within the specified time, the node is added to BadNodes.

When a reply Reply is received from the gen_server process at a node Node, {Node,Reply} is added to Replies. Reply is passed from the return value of Module:handle_call/3.

Warning

If one of the nodes cannot process monitors, for example, C or Java nodes, and the gen_server process is not started when the requests are sent, but starts within 2 seconds, this function waits the whole Timeout, which may be infinity.

This problem does not exist if all nodes are Erlang nodes.

To prevent late answers (after the time-out) from polluting the message queue of the caller, a middleman process is used to do the calls. Late answers are then discarded when they arrive to the terminated middleman process.

receive_response(ReqId, Timeout)

(since OTP 24.0)

-spec receive_response(ReqId, Timeout) -> Result
                          when
                              ReqId :: request_id(),
                              Timeout :: response_timeout(),
                              Response ::
                                  {reply, Reply :: term()} | {error, {Reason :: term(), server_ref()}},
                              Result :: Response | timeout.

Receive a request response.

Receive a response corresponding to the request identifier ReqId. The request must have been made by send_request/2, and it must have been made by the same process calling this function.

Timeout specifies how long to wait for a response. If no response is received within the specified time, this function returns timeout. Assuming that the server executes on a node supporting aliases (introduced in OTP 24) the request will also be abandoned. That is, no response will be received after a time-out. Otherwise, a stray response might be received at a later time.

The return value Reply is passed from the return value of Module:handle_call/3.

The function returns an error if the gen_server died before a reply was sent.

The difference between receive_response/2 and wait_response/2 is that receive_response/2 abandons the request at time-out so that a potential future response is ignored, while wait_response/2 does not.

receive_response(ReqIdCollection, Timeout, Delete)

(since OTP 25.0)

-spec receive_response(ReqIdCollection, Timeout, Delete) -> Result
                          when
                              ReqIdCollection :: request_id_collection(),
                              Timeout :: response_timeout(),
                              Delete :: boolean(),
                              Response ::
                                  {reply, Reply :: term()} | {error, {Reason :: term(), server_ref()}},
                              Result ::
                                  {Response,
                                   Label :: term(),
                                   NewReqIdCollection :: request_id_collection()} |
                                  no_request | timeout.

Receive a request response in a collection.

Receive a response in ReqIdCollection. All request identifiers of ReqIdCollection must correspond to requests that have been made using send_request/2 or send_request/4, and all requests must have been made by the process calling this function.

The Label in the response is the Label associated with the request identifier that the response corresponds to. The Label of a request identifier is associated when adding the request id to a collection, or when sending the request using send_request/4.

Compared to receive_response/2, the returned result or exception associated with a specific request identifier will be wrapped in a 3-tuple {Response, Label, NewReqIdCollection}. Response is the value that would have been produced by receive_response/2, Label is the value associated with the specific request identifier and NewReqIdCollection is a possibly modified request identifier collection.

If ReqIdCollection is empty, no_request will be returned.

Timeout specifies how long to wait for a response. If no response is received within the specified time, the function returns timeout. Assuming that the server executes on a node supporting aliases (introduced in OTP 24) all requests identified by ReqIdCollection will also be abandoned. That is, no responses will be received after a time-out. Otherwise, stray responses might be received at a later time.

The difference between receive_response/3 and wait_response/3 is that receive_response/3 abandons the requests at time-out so that potential future responses are ignored, while wait_response/3 does not.

If Delete is true, the association with Label is deleted from ReqIdCollection in the resulting NewReqIdCollection. If Delete is false, NewReqIdCollection will equal ReqIdCollection. Note that deleting an association is not for free and that a collection containing already handled requests can still be used by subsequent calls to receive_response/3, check_response/3, and wait_response/3.

However, without deleting handled associations, the above calls will not be able to detect when there are no more outstanding requests to handle, so you will have to keep track of this some other way than relying on a no_request return. Note that if you pass a collection only containing associations of already handled or abandoned requests to this function, it will always block until Timeout expires and then return timeout.

reply(Client, Reply)

-spec reply(Client :: from(), Reply :: term()) -> ok.

Send a reply to a client.

This function can be used by a gen_server process to explicitly send a reply to a client that called call/2,3 or multi_call/2,3,4, when the reply cannot be passed in the return value of Module:handle_call/3.

Client must be the From argument provided to the handle_call/3 callback function. Reply is any term passed back to the client as the return value of call/2,3 or multi_call/2,3,4.

reqids_add(ReqId, Label, ReqIdCollection)

(since OTP 25.0)

-spec reqids_add(ReqId :: request_id(), Label :: term(), ReqIdCollection :: request_id_collection()) ->
                    NewReqIdCollection :: request_id_collection().

Store a request identifier in a colletion.

Stores ReqId and associates a Label with the request identifier by adding this information to ReqIdCollection and returning the resulting request identifier collection.

reqids_new()

(since OTP 25.0)

-spec reqids_new() -> NewReqIdCollection :: request_id_collection().

Create an empty request identifier collection.

Returns a new empty request identifier collection. A request identifier collection can be utilized to handle multiple outstanding requests.

Request identifiers of requests made by send_request/2 can be stored in a collection using reqids_add/3. Such a collection of request identifiers can later be used in order to get one response corresponding to a request in the collection by passing the collection as argument to receive_response/3, wait_response/3, or, check_response/3.

reqids_size/1 can be used to determine the number of request identifiers in a collection.

reqids_size(ReqIdCollection)

(since OTP 25.0)

-spec reqids_size(ReqIdCollection :: request_id_collection()) -> non_neg_integer().

Returns the number of request identifiers in ReqIdCollection.

reqids_to_list(ReqIdCollection)

(since OTP 25.0)

-spec reqids_to_list(ReqIdCollection :: request_id_collection()) ->
                        [{ReqId :: request_id(), Label :: term()}].

Convert a request identifier collection to a list.

Returns a list of {ReqId, Label} tuples which corresponds to all request identifiers with their associated labels in ReqIdCollection.

send_request(ServerRef, Request)

(since OTP 23.0)

-spec send_request(ServerRef :: server_ref(), Request :: term()) -> ReqId :: request_id().

Send an asynchronous call request.

Sends Request to the gen_server process identified by ServerRef and returns a request identifier ReqId.

The return value ReqId shall later be used with receive_response/2, wait_response/2, or check_response/2 to fetch the actual result of the request. Besides passing the request identifier directly to these functions, it can also be stored in a request identifier collection using reqids_add/3. Such a collection of request identifiers can later be used in order to get one response corresponding to a request in the collection by passing the collection as argument to receive_response/3, wait_response/3, or check_response/3. If you are about to store the request identifier in a collection, you may want to consider using send_request/4 instead.

The call gen_server:receive_response(gen_server:send_request(ServerRef, Request), Timeout) can be seen as equivalent to gen_server:call(ServerRef, Request, Timeout), ignoring the error handling.

The gen_server process calls Module:handle_call/3 to handle the request.

See the type server_ref/0 for the possible values for ServerRef.

Request is any term that is passed as the first argument to Module:handle_call/3.

send_request(ServerRef, Request, Label, ReqIdCollection)

(since OTP 25.0)

-spec send_request(ServerRef :: server_ref(),
                   Request :: term(),
                   Label :: term(),
                   ReqIdCollection :: request_id_collection()) ->
                      NewReqIdCollection :: request_id_collection().

Send an asynchronous call request and add it to a request identifier collection.

Sends Request to the gen_server process identified by ServerRef. The Label will be associated with the request identifier of the operation and added to the returned request identifier collection NewReqIdCollection. The collection can later be used in order to get one response corresponding to a request in the collection by passing the collection as argument to receive_response/3, wait_response/3, or check_response/3.

The same as calling reqids_add(send_request(ServerRef, Request), Label, ReqIdCollection), but slightly more efficient.

start(Module, Args, Options)

-spec start(Module :: module(), Args :: term(), Options :: [start_opt()]) -> start_ret().

Start a server, neither linked nor registered.

Equivalent to start/4 except that the gen_server process is not registered with any name service.

start(ServerName, Module, Args, Options)

-spec start(ServerName :: server_name(), Module :: module(), Args :: term(), Options :: [start_opt()]) ->
               start_ret().

Start a server, registered but not linked.

Creates a standalone gen_server process, that is, a gen_server process that is not part of a supervision tree, and thus has no supervisor.

Other than that see start_link/4.

start_link(Module, Args, Options)

-spec start_link(Module :: module(), Args :: term(), Options :: [start_opt()]) -> start_ret().

Start a server, linked but not registered.

Equivalent to start_link/4 except that the gen_server process is not registered with any name service.

start_link(ServerName, Module, Args, Options)

-spec start_link(ServerName :: server_name(),
                 Module :: module(),
                 Args :: term(),
                 Options :: [start_opt()]) ->
                    start_ret().

Start a server, linked and registered.

Creates a gen_server process as part of a supervision tree. This function is to be called, directly or indirectly, by the supervisor. For example, it ensures that the gen_server process is spawned as linked to the caller (supervisor).

The gen_server process calls Module:init/1 to initialize. To ensure a synchronized startup procedure, start_link/3,4 does not return until Module:init/1 has returned or failed.

ServerName specifies with what name and now to register the server name. See type server_name/0 for different name registrations.

Module is the name of the callback module.

Args is any term that is passed as the argument to Module:init/1.

See type start_opt/0 for Options for starting the gen_server process.

See type start_ret/0 for a description this function's return values.

If start_link/3,4 returns ignore or {error, _}, the started gen_server process has terminated. If an 'EXIT' message was delivered to the calling process (due to the process link), that message has been consumed.

Warning

Before OTP 26.0, if the started gen_server process returned e.g. {stop, Reason} from Module:init/1, this function could return {error, Reason} before the started gen_server process had terminated so starting again might fail because VM resources such as the registered name was not yet unregistered. An 'EXIT' message could arrive later to the process calling this function.

But if the started gen_server process instead failed during Module:init/1, a process link {'EXIT', Pid, Reason} message caused this function to return {error, Reason}, so the 'EXIT' message had been consumed and the started gen_server process had terminated.

Since it was impossible to tell the difference between these two cases from start_link/3,4's return value, this inconsistency was cleaned up in OTP 26.0.

The difference between returning {stop, _} and {error, _} from Module:init/1, is that {error, _} results in a graceful ("silent") termination since the gen_server process exits with reason normal.

start_monitor(Module, Args, Options)

(since OTP 23.0)

-spec start_monitor(Module :: module(), Args :: term(), Options :: [start_opt()]) -> start_mon_ret().

Start a server, monitored but neither linked nor registered.

Equivalent to start_monitor/4 except that the gen_server process is not registered with any name service.

start_monitor(ServerName, Module, Args, Options)

(since OTP 23.0)

-spec start_monitor(ServerName :: server_name(),
                    Module :: module(),
                    Args :: term(),
                    Options :: [start_opt()]) ->
                       start_mon_ret().

Start a server, monitored and registered, but not linked.

Creates a standalone gen_server process, that is, a gen_server process that is not part of a supervision tree (and thus has no supervisor) and atomically sets up a monitor to the newly created server.

Other than that see start_link/3,4. Note that the return value for a successful start differs in that it returns a monitor reference. See type start_mon_ret/0.

If the start is not successful, the caller will be blocked until the monitor's 'DOWN' message has been received and removed from the message queue.

stop(ServerRef)

(since OTP 18.0)

-spec stop(ServerRef :: server_ref()) -> ok.

Equivalent to stop(ServerRef, normal, infinity).

stop(ServerRef, Reason, Timeout)

(since OTP 18.0)

-spec stop(ServerRef :: server_ref(), Reason :: term(), Timeout :: timeout()) -> ok.

Stop a server.

Orders the generic server specified by ServerRef to exit with the specified Reason and waits for it to terminate. The gen_server process calls Module:terminate/2 before exiting.

The function returns ok if the server terminates with the expected reason. Any other reason than normal, shutdown, or {shutdown,Term} causes an error report to be issued using logger. An exit signal with the same reason is sent to linked processes and ports.

Timeout is an integer that specifies how many milliseconds to wait for the server to terminate, or the atom infinity to wait indefinitely. If the server has not terminated within the specified time, the call exits the calling process with reason timeout.

If the process does not exist, the call exits the calling process with reason noproc, or with reason {nodedown,Node} if the connection fails to the remote Node where the server runs.

wait_response(ReqId, WaitTime)

(since OTP 23.0)

-spec wait_response(ReqId, WaitTime) -> Result
                       when
                           ReqId :: request_id(),
                           WaitTime :: response_timeout(),
                           Response ::
                               {reply, Reply :: term()} | {error, {Reason :: term(), server_ref()}},
                           Result :: Response | timeout.

Wait for a request response.

Wait for the response to the request identifier ReqId. The request must have been made by send_request/2, and it must have been made by the same process calling this function.

WaitTime specifies how long to wait for a reply. If no reply is received within the specified time, the function returns timeout and no cleanup is done. Thus the function can be invoked repeatedly until a reply is returned.

The return value Reply is passed from the return value of Module:handle_call/3.

The function returns an error if the gen_server died before a reply was sent.

The difference between receive_response/2 and wait_response/2 is that receive_response/2 abandons the request at time-out so that a potential future response is ignored, while wait_response/2 does not.

wait_response(ReqIdCollection, WaitTime, Delete)

(since OTP 25.0)

-spec wait_response(ReqIdCollection, WaitTime, Delete) -> Result
                       when
                           ReqIdCollection :: request_id_collection(),
                           WaitTime :: response_timeout(),
                           Delete :: boolean(),
                           Response ::
                               {reply, Reply :: term()} | {error, {Reason :: term(), server_ref()}},
                           Result ::
                               {Response,
                                Label :: term(),
                                NewReqIdCollection :: request_id_collection()} |
                               no_request | timeout.

Wait for any request response in a collection.

Wait for a response in a ReqIdCollection. All request identifiers of ReqIdCollection must correspond to requests that have been made using send_request/2 or send_request/4, and all requests must have been made by the process calling this function.

The Label in the response is the Label associated with the request identifier that the response corresponds to. The Label of a request identifier is associated when adding the request id to a collection, or when sending the request using send_request/4.

Compared to wait_response/2, the returned result or exception associated with a specific request identifier will be wrapped in a 3-tuple {Response, Label, NewReqIdCollection}. Response is the value that would have been produced by wait_response/2, Label is the value associated with the specific request identifier and NewReqIdCollection is a possibly modified request identifier collection.

If ReqIdCollection is empty, no_request will be returned.

If no response is received before WaitTime has expired, timeout is returned. It is valid to continue waiting for a response as many times as needed up until a response has been received and completed by check_response(), receive_response(), or wait_response().

The difference between receive_response/3 and wait_response/3 is that receive_response/3 abandons requests at time-out so that potential future responses are ignored, while wait_response/3 does not.

If Delete is true, the association with Label has been deleted from ReqIdCollection in the resulting NewReqIdCollection. If Delete is false, NewReqIdCollection will equal ReqIdCollection. Note that deleting an association is not for free and that a collection containing already handled requests can still be used by subsequent calls to wait_response/3, check_response/3, and receive_response/3.

However, without deleting handled associations, the above calls will not be able to detect when there are no more outstanding requests to handle, so you will have to keep track of this some other way than relying on a no_request return. Note that if you pass a collection only containing associations of already handled or abandoned requests to this function, it will always block until WaitTime expires and then return timeout.

gen_statem behaviour

Generic state machine behavior.

gen_statem provides a generic state machine behaviour that since Erlang/OTP 20.0 replaces its predecessor gen_fsm, and should be used for new code. The gen_fsm behaviour remains in OTP "as is" to not break old code using it.

A generic state machine server process (gen_statem) implemented using this module has a standard set of interface functions and includes functionality for tracing and error reporting. It also fits into an OTP supervision tree. For more information, see gen_statem section in OTP Design Principles.

Note

If you are new to gen_statem and want an overview of concepts and operation the section gen_statem Behaviour located in the User's Guide OTP Design Principles is recommended to read. This reference manual focuses on being correct and complete, which might make it hard to see the forest for all the trees.

Features

gen_statem has got the same features that gen_fsm had and adds some really useful:

Co-located state code

Arbitrary term state

Event postponing

Self-generated events

State time-out

Multiple generic named time-outs

Absolute time-out time

Automatic state enter calls

Reply from other state than the request, traceable with sys

Multiple replies, traceable with sys

Changing the callback module

Two callback modes are supported:

state_functions - for finite-state machines (gen_fsm like), which requires the state to be an atom and uses that state as the name of the current callback function, arity 3.

handle_event_function - that allows the state to be any term and that uses handle_event/4 as callback function for all states.

The callback modes for gen_statem differs from the one for gen_fsm, but it is still fairly easy to rewrite from gen_fsm to gen_statem. See the rewrite guide at the start of the gen_fsm documentation.

Callback module

A gen_statem assumes all specific parts to be located in a callback module exporting a predefined set of functions. The relationship between the behavior functions and the callback functions is as follows:

gen_statem module            Callback module
-----------------            ---------------
gen_statem:start
gen_statem:start_monitor
gen_statem:start_link -----> Module:init/1

Server start or code change
                      -----> Module:callback_mode/0
                      selects callback mode

gen_statem:stop
Supervisor exit
Callback failure      -----> Module:terminate/3

gen_statem:call
gen_statem:cast
gen_statem:send_request
erlang:send
erlang:'!'            -----> Module:StateName/3
                   or -----> Module:handle_event/4
                   depending on callback mode

Release upgrade/downgrade
(code change)
                      -----> Module:code_change/4

State callback

The state callback for a specific state in a gen_statem is the callback function that is called for all events in this state. It is selected depending on which callback mode that the callback module defines with the callback function Module:callback_mode/0.

When the callback mode is state_functions, the state must be an atom and is used as the state callback name; see Module:StateName/3. This co-locates all code for a specific state in one function as the gen_statem engine branches depending on state name. Note the fact that the callback function Module:terminate/3 makes the state name terminate unusable in this mode.

When the callback mode is handle_event_function, the state can be any term and the state callback name is Module:handle_event/4. This makes it easy to branch depending on state or event as you desire. Be careful about which events you handle in which states so that you do not accidentally postpone an event forever creating an infinite busy loop.

Event types

Events are of different types, therefore the callback functions can know the origin of an event when handling it. External events are call, cast, and info. Internal events are timeout and internal.

Event handling

When gen_statem receives a process message it is transformed into an event and the state callback is called with the event as two arguments: type and content. When the state callback has processed the event it returns to gen_statem which does a state transition. If this state transition is to a different state, that is: NextState =/= State, it is a state change.

Transition actions

The state callback may return transition actions for gen_statem to execute during the state transition, for example to set a time-out or reply to a call.

Reply to a call

See gen_statem:call/2,3 about how to reply to a call. A reply can be sent from any state callback, not just the one that got the request event.

Event postponing

One of the possible transition actions is to postpone the current event. Then it will not be handled in the current state. The gen_statem engine keeps a queue of events divided into postponed events and events still to process (not presented yet). After a state change the queue restarts with the postponed events.

The gen_statem event queue model is sufficient to emulate the normal process message queue with selective receive. Postponing an event corresponds to not matching it in a receive statement, and changing states corresponds to entering a new receive statement.

Event insertion

The state callback can insert events using the transition action next_event, and such an event is inserted in the event queue as the next to call the state callback with. That is, as if it is the oldest incoming event. A dedicated event_type/0 internal can be used for such events making it possible to safely distinguish them from external events.

Inserting an event replaces the trick of calling your own state handling functions that you often would have to resort to in, for example, gen_fsm to force processing an inserted event before others.

Note

If you postpone an event and (against good practice) directly call a different state callback, the postponed event is not retried, since there was no state change.

Instead of directly calling a state callback, do a state change. This makes the gen_statem engine retry postponed events.

Inserting an event in a state change also triggers the new state callback to be called with that event before receiving any external events.

State enter calls

The gen_statem engine can automatically make a special call to the state callback whenever a new state is entered; see state_enter/0. This is for writing code common to all state entries. Another way to do it is to explicitly insert an event at the state transition, and/or to use a dedicated state transition function, but that is something you will have to remember at every state transition to the state(s) that need it.

For the details of a state transition, see type transition_option/0.

Hibernation

The gen_statem process can go into hibernation; see proc_lib:hibernate/3. It is done when a state callback or Module:init/1 specifies hibernate in the returned Actions list. This feature can be useful to reclaim process heap memory while the server is expected to be idle for a long time. However, use it with care, as hibernation can be too costly to use after every event; see erlang:hibernate/3.

There is also a server start option {hibernate_after, Timeout} for start/3,4, start_link/3,4, start_monitor/3,4, or enter_loop/4,5,6, that may be used to automatically hibernate the server.

Callback failure

If a callback function fails or returns a bad value, the gen_statem terminates. However, an exception of class throw is not regarded as an error but as a valid return, from all callback functions.

System messages and the sys module

A gen_statem handles system messages as described in sys. The sys module can be used for debugging a gen_statem. Replies sent through transition actions gets logged, but not replies sent through reply/1,2.

Trapping exit

A gen_statem process, like all gen_* behaviours, does not trap exit signals automatically; this must be explicitly initiated in the callback module (by calling process_flag(trap_exit, true) preferably from init/1.

Server termination

If the gen_statem process terminates, e.g. as a result of a callback function returning {stop, Reason}, an exit signal with this Reason is sent to linked processes and ports. See Processes in the Reference Manual for details regarding error handling using exit signals.

Note

For some important information about distributed signals, see the Blocking Signaling Over Distribution section in the Processes chapter of the Erlang Reference Manual. Blocking signaling can, for example, cause call time-outs in gen_statem to be significantly delayed.

Bad argument

Unless otherwise stated, all functions in this module fail if the specified gen_statem does not exist or if bad arguments are specified.

Example

The following example shows a simple pushbutton model for a toggling pushbutton implemented with callback mode state_functions. You can push the button and it replies if it went on or off, and you can ask for a count of how many times it has been pushed to switch on.

Pushbutton State Diagram

---
title: Pushbutton State Diagram
---
stateDiagram-v2
    [*]  --> off
    off  --> on  : push\n* Increment count\n* Reply 'on'
    on   --> off : push\n* Reply 'off'

Not shown in the state diagram:

The API function push() generates an event push of type call.

The API function get_count() generates an event get_count of type call that is handled in all states by replying with the current count value.

Unknown events are ignored and discarded.

There is boilerplate code for start, stop, terminate, code change, init, to set the callback mode to state_functions, etc...

Pushbutton Code

The following is the complete callback module file pushbutton.erl:

-module(pushbutton).
-behaviour(gen_statem).

-export([start/0,push/0,get_count/0,stop/0]).
-export([terminate/3,code_change/4,init/1,callback_mode/0]).
-export([on/3,off/3]).

name() -> pushbutton_statem. % The registered server name

%% API.  This example uses a registered name name()
%% and does not link to the caller.
start() ->
    gen_statem:start({local,name()}, ?MODULE, [], []).
push() ->
    gen_statem:call(name(), push).
get_count() ->
    gen_statem:call(name(), get_count).
stop() ->
    gen_statem:stop(name()).

%% Mandatory callback functions
terminate(_Reason, _State, _Data) ->
    void.
code_change(_Vsn, State, Data, _Extra) ->
    {ok,State,Data}.
init([]) ->
    %% Set the initial state + data.  Data is used only as a counter.
    State = off, Data = 0,
    {ok,State,Data}.
callback_mode() -> state_functions.

%%% state callback(s)

off({call,From}, push, Data) ->
    %% Go to 'on', increment count and reply
    %% that the resulting status is 'on'
    {next_state,on,Data+1,[{reply,From,on}]};
off(EventType, EventContent, Data) ->
    handle_event(EventType, EventContent, Data).

on({call,From}, push, Data) ->
    %% Go to 'off' and reply that the resulting status is 'off'
    {next_state,off,Data,[{reply,From,off}]};
on(EventType, EventContent, Data) ->
    handle_event(EventType, EventContent, Data).

%% Handle events common to all states
handle_event({call,From}, get_count, Data) ->
    %% Reply with the current count
    {keep_state,Data,[{reply,From,Data}]};
handle_event(_, _, Data) ->
    %% Ignore all other events
    {keep_state,Data}.

The following is a shell session when running it:

1> pushbutton:start().
{ok,<0.36.0>}
2> pushbutton:get_count().
0
3> pushbutton:push().
on
4> pushbutton:get_count().
1
5> pushbutton:push().
off
6> pushbutton:get_count().
1
7> pushbutton:stop().
ok
8> pushbutton:push().
** exception exit: {noproc,{gen_statem,call,[pushbutton_statem,push,infinity]}}
     in function  gen:do_for_proc/2 (gen.erl, line 261)
     in call from gen_statem:call/3 (gen_statem.erl, line 386)

To compare styles, here follows the same example using callback mode handle_event_function, or rather, the code to replace after function init/1 of the pushbutton.erl example file above:

callback_mode() -> handle_event_function.

%%% state callback(s)

handle_event({call,From}, push, off, Data) ->
    %% Go to 'on', increment count and reply
    %% that the resulting status is 'on'
    {next_state,on,Data+1,[{reply,From,on}]};
handle_event({call,From}, push, on, Data) ->
    %% Go to 'off' and reply that the resulting status is 'off'
    {next_state,off,Data,[{reply,From,off}]};
%%
%% Event handling common to all states
handle_event({call,From}, get_count, State, Data) ->
    %% Reply with the current count
    {next_state,State,Data,[{reply,From,Data}]};
handle_event(_, _, State, Data) ->
    %% Ignore all other events
    {next_state,State,Data}.

Note

API changes

This behavior appeared in Erlang/OTP 19.0 as experimental.

In OTP 19.1 a backwards incompatible change of the return tuple from Module:init/1 was made, the mandatory callback function Module:callback_mode/0 was introduced, and enter_loop/4 was added.

In OTP 19.2 state enter calls were added.

In OTP 19.3 state time-outs were added.

In OTP 20.0 generic time-outs were added and gen_statem was stated to be no longer experimental and preferred over gen_fsm.

In OTP 22.1 time-out content update and explicit time-out cancel were added.

In OTP 22.3 the possibility to change the callback module with actions change_callback_module, push_callback_module and pop_callback_module, was added.

In OTP 23.0 start_monitor/3,4 were added, as well as functions for asynchronous calls: send_request/2, wait_response/1,2, and check_response/2.

In OTP 24.0 receive_response/1,2 were added.

In OTP 25.0 Module:format_status/1 was added to replace Module:format_status/2, as well as functions for collections of asynchronous calls: send_request/4, wait_response/3, receive_response/3, check_response/3, reqids_new/0, reqids_size/1, reqids_add/3, reqids_to_list/1.

In OTP 26.0 the possibility to return {error, Reason} from Module:init/1 was added.

In OTP 27.0 Module:format_status/2 was deprecated.

See Also

gen_event, gen_fsm, gen_server, proc_lib, supervisor, sys.

Summary

Types

action()

Actions for a state transition, or when starting the server.

callback_mode()

One function per state or one common event handler.

callback_mode_result()

Return value from Module:callback_mode/0.

data()

Generic state data for the server.

enter_action()

Actions for any callback: hibernate, time-outs or replies.

enter_loop_opt()

Server start options for the enter_loop/4,5,6, start/3,4, start_link/3,4, and start_monitor/3,4, functions.

event_content()

Event payload from the event's origin, delivered to the state callback.

event_handler_result(StateType)

event_handler_result(StateType, DataType)

Return value from a state callback after handling an event.

event_timeout()

How long to wait for an event.

event_type()

All event types: external, time-out, or internal.

external_event_type()

Event from a call, cast, or regular process message; "info".

format_status()

A map that describes the server's status.

from()

A call event's reply destination.

generic_timeout()

How long to wait for a named time-out event.

hibernate()

Hibernate the server process.

init_result(StateType)

init_result(StateType, DataType)

The return value from Module:init/1.

postpone()

Postpone an event to handle it later.

reply_action()

Reply to a call/2,3.

reply_tag()

A handle that associates a reply to the corresponding request.

request_id()

An opaque request identifier. See send_request/2 for details.

request_id_collection()

An opaque collection of request identifiers (request_id/0).

response_timeout()

Response time-out for an asynchronous call.

server_name()

Server name specification: local, global, or via registered.

server_ref()

Server specification: pid/0 or registered server_name/0.

start_mon_ret()

Return value from the start_monitor/3,4 functions.

start_opt()

Server start options for the start/3,4, start_link/3,4, and start_monitor/3,4 functions.

start_ret()

Return value from the start/3,4 and start_link/3,4 functions.

state()

State name or state term.

state_callback_result(ActionType, DataType)

Return value from any state callback.

state_enter()

Callback mode modifier for state enter calls: the atom state_enter.

state_enter_result(State)

state_enter_result(State, DataType)

Return value from a state callback after a state enter call.

state_name()

State name in callback mode state_functions.

state_timeout()

How long to wait in the current state.

timeout_action()

Event time-out, generic time-outs or state time-out.

timeout_cancel_action()

Clearer way to cancel a time-out than the original setting it to 'infinity'.

timeout_event_type()

Event time-out, generic time-out, or state time-out.

timeout_option()

Time-out timer start option, to select absolute time of expiry.

timeout_update_action()

Update the EventContent without affecting the time of expiry.

transition_option()

State transition options set by actions.

Callbacks

callback_mode()

Select the callback mode and possibly state enter calls.

code_change(OldVsn, OldState, OldData, Extra)

Update the state and data after code change.

format_status(Status)

Format/limit the status value.

format_status/2 deprecated

Format/limit the status value.

handle_event/4

State callback in callback mode handle_event_function.

init(Args)

Initialize the state machine.

'StateName'/3

State callback in callback mode state_functions.

terminate/3

Handle state machine termination.

Functions

call(ServerRef, Request)

Equivalent to call(ServerRef, Request, infinity).

call(ServerRef, Request, Timeout)

Call a server: send request and wait for response.

cast(ServerRef, Msg)

Cast an event to a server.

check_response(Msg, ReqId)

Check if a received message is a request response.

check_response(Msg, ReqIdCollection, Delete)

Check if a received message is a request response in a collection.

enter_loop(Module, Opts, State, Data)

Equivalent to enter_loop(Module, Opts, State, Data, self(), []).

enter_loop(Module, Opts, State, Data, Server_or_Actions)

Make the calling process become a gen_statem server.

enter_loop(Module, Opts, State, Data, Server, Actions)

Make the calling process become a gen_statem server.

receive_response(ReqId)

Equivalent to receive_response(ReqId, infinity).

receive_response(ReqId, Timeout)

Receive a request response.

receive_response(ReqIdCollection, Timeout, Delete)

Receive a request response in a collection.

reply(Replies)

Send one or multiple call replies.

reply(From, Reply)

Send a call Reply to From.

reqids_add(ReqId, Label, ReqIdCollection)

Store a request identifier in a colletion.

reqids_new()

Create an empty request identifier collection.

reqids_size(ReqIdCollection)

Return the number of request identifiers in ReqIdCollection.

reqids_to_list(ReqIdCollection)

Convert a request identifier collection to a list.

send_request(ServerRef, Request)

Send an asynchronous call request.

send_request(ServerRef, Request, Label, ReqIdCollection)

Send an asynchronous call request and add it to a request identifier collection.

start(Module, Args, Opts)

Start a server, neither linked nor registered.

start(ServerName, Module, Args, Opts)

Start a server, registered but not linked.

start_link(Module, Args, Opts)

Start a server, linked but not registered.

start_link(ServerName, Module, Args, Opts)

Start a server, linked and registered.

start_monitor(Module, Args, Opts)

Start a server, monitored but neither linked nor registered.

start_monitor(ServerName, Module, Args, Opts)

Start a server, monitored and registered, but not linked.

stop(ServerRef)

Equivalent to stop(ServerRef, normal, infinity).

stop(ServerRef, Reason, Timeout)

Stop a server.

wait_response(ReqId)

Equivalent to wait_response(ReqId, infinity).

wait_response(ReqId, WaitTime)

Wait for a request response.

wait_response(ReqIdCollection, WaitTime, Delete)

Wait for any request response in a collection.

Types

action()

(since OTP 19.0)

-type action() ::
          postpone |
          {postpone, Postpone :: postpone()} |
          {next_event, EventType :: event_type(), EventContent :: event_content()} |
          {change_callback_module, NewModule :: module()} |
          {push_callback_module, NewModule :: module()} |
          pop_callback_module |
          enter_action().

Actions for a state transition, or when starting the server.

These transition actions can be invoked by returning them from the state callback when it is called with an event, from Module:init/1 or by passing them to enter_loop/4,5,6. They are not allowed from state enter calls.

Actions are executed in the containing list order.

Actions that set transition options override any previous of the same type, so the last in the containing list wins. For example, the last postpone/0 overrides any previous postpone/0 in the list.

{postpone, Value} - Sets the transition_option() postpone/0 for this state transition. This action is ignored when returned from Module:init/1 or passed to enter_loop/4,5,6, as there is no event to postpone in those cases.

postpone is equivalent to {postpone, true}.

{next_event, EventType, EventContent} - This action does not set any transition_option() but instead stores the specified EventType and EventContent for insertion after all actions have been executed.

The stored events are inserted in the queue as the next to process before any already queued events. The order of these stored events is preserved, so the first next_event in the containing list becomes the first to process.

An event of type internal should be used when you want to reliably distinguish an event inserted this way from any external event.

{change_callback_module, NewModule} - Changes the callback module to NewModule which will be used when calling all subsequent state callbacks.
Since OTP 22.3.

The gen_statem engine will find out the callback mode of NewModule by calling NewModule:callback_mode/0 before the next state callback.

Changing the callback module does not affect the state transition in any way, it only changes which module that handles the events. Be aware that all relevant callback functions in NewModule such as the state callback, NewModule:code_change/4, NewModule:format_status/1 and NewModule:terminate/3 must be able to handle the state and data from the old module.

{push_callback_module, NewModule} - Pushes the current callback module to the top of an internal stack of callback modules, and changes the callback module to NewModule. Otherwise like {change_callback_module, NewModule} above.
Since OTP 22.3.

pop_callback_module - Pops the top module from the internal stack of callback modules and changes the callback module to be the popped module. If the stack is empty the server fails. Otherwise like {change_callback_module, NewModule} above.
Since OTP 22.3.

callback_mode()

(not exported) (since OTP 19.1)

-type callback_mode() :: state_functions | handle_event_function.

One function per state or one common event handler.

The callback mode is selected with the return value from Module:callback_mode/0:

state_functions - The state must be of type state_name/0 and one callback function per state, that is, Module:StateName/3, is used.

handle_event_function - The state can be any term and the callback function Module:handle_event/4 is used for all states.

The function Module:callback_mode/0 is called when starting the gen_statem, after code change and after changing the callback module with any of the actions change_callback_module, push_callback_module, or pop_callback_module. The result is cached for subsequent calls to state callbacks.

callback_mode_result()

(since OTP 19.2)

-type callback_mode_result() :: callback_mode() | [callback_mode() | state_enter()].

Return value from Module:callback_mode/0.

This is the return type from Module:callback_mode/0 which selects callback mode and whether to do state enter calls, or not.

data()

(not exported) (since OTP 19.0)

-type data() :: term().

Generic state data for the server.

A term in which the state machine implementation is to store any server data it needs. The difference between this and the state/0 itself is that a change in this data does not cause postponed events to be retried. Hence, if a change in this data would change the set of events that are handled, then that data item should be part of the state/0 instead.

enter_action()

(since OTP 19.0)

-type enter_action() ::
          hibernate | {hibernate, Hibernate :: hibernate()} | timeout_action() | reply_action().

Actions for any callback: hibernate, time-outs or replies.

These transition actions are allowed when a action/0 is allowed, and also from a state enter call, and can be invoked by returning them from the state callback, from Module:init/1 or by passing them to enter_loop/4,5,6.

Actions are executed in the containing list order.

Actions that set transition options override any previous of the same type, so the last in the containing list wins. For example, the last event_timeout/0 overrides any previous event_timeout/0 in the list.

{hibernate, Value} - Sets the transition_option/0 hibernate/0 for this state transition.

hibernate is equivalent to {hibernate, true}.

enter_loop_opt()

(since OTP 19.0)

-type enter_loop_opt() ::
          {hibernate_after, HibernateAfterTimeout :: timeout()} | {debug, Dbgs :: [sys:debug_option()]}.

Server start options for the enter_loop/4,5,6, start/3,4, start_link/3,4, and start_monitor/3,4, functions.

See start_link/4.

event_content()

(not exported) (since OTP 19.0)

-type event_content() :: term().

Event payload from the event's origin, delivered to the state callback.

See event_type that describes the origins of the different event types, which is also where the event's content comes from.

event_handler_result(StateType)

(since OTP 19.0)

-type event_handler_result(StateType) :: event_handler_result(StateType, term()).

event_handler_result(StateType, DataType)

(since OTP 19.0)

-type event_handler_result(StateType, DataType) ::
          {next_state, NextState :: StateType, NewData :: DataType} |
          {next_state, NextState :: StateType, NewData :: DataType, Actions :: [action()] | action()} |
          state_callback_result(action(), DataType).

Return value from a state callback after handling an event.

StateType is state_name/0 if callback mode is state_functions, or state/0 if callback mode is handle_event_function.

{next_state, NextState, NewData [, Actions]} - The gen_statem does a state transition to NextState (which may be the same as the current state), sets NewData as the current server data/0, and executes all Actions. If NextState =/= CurrentState the state transition is a state change.

event_timeout()

(not exported) (since OTP 19.0)

-type event_timeout() :: Time :: timeout() | integer().

How long to wait for an event.

Starts a timer set by timeout_action/0 Time, or {timeout, Time, EventContent [, Options]}.

When the timer expires an event of event_type/0 timeout will be generated. See erlang:start_timer/4 for how Time and Options are interpreted. Future erlang:start_timer/4 Options will not necessarily be supported.

Any event that arrives cancels this time-out. Note that a retried or inserted event counts as arrived. So does a state time-out zero event, if it was generated before this time-out is requested.

If Time is infinity, no timer is started, as it never would expire anyway.

If Time is relative and 0 no timer is actually started, instead the the time-out event is enqueued to ensure that it gets processed before any not yet received external event, but after already queued events.

Note that it is not possible nor needed to cancel this time-out, as it is cancelled automatically by any other event, meaning that whenever a callback is invoked that may want to cancel this time-out, the timer is already cancelled or expired.

The timer EventContent can be updated with the {timeout, update, NewEventContent} action without affecting the time of expiry.

event_type()

(since OTP 19.0)

-type event_type() :: external_event_type() | timeout_event_type() | internal.

All event types: external, time-out, or internal.

internal events can only be generated by the state machine itself through the transition action next_event.

external_event_type()

(not exported) (since OTP 19.0)

-type external_event_type() :: {call, From :: from()} | cast | info.

Event from a call, cast, or regular process message; "info".

Type {call, From} originates from the API functions call/2,3 or send_request/2. The event contains From, which is whom to reply to by a reply_action/0 or reply/2,3 call.

Type cast originates from the API function cast/2.

Type info originates from regular process messages sent to the gen_statem process.

format_status()

(since OTP 19.0)

-type format_status() ::
          #{state => state(),
            data => data(),
            reason => term(),
            queue => [{event_type(), event_content()}],
            postponed => [{event_type(), event_content()}],
            timeouts => [{timeout_event_type(), event_content()}],
            log => [sys:system_event()]}.

A map that describes the server's status.

The keys are:

state - The current state.

data - The state data.

reason - The reason that caused the process to terminate.

queue - The event queue.

postponed - The queue of postponed events.

timeouts - The active time-outs.

log - The sys log of the server.

New associations may be added to the status map without prior notice.

from()

(since OTP 19.0)

-type from() :: {To :: pid(), Tag :: reply_tag()}.

A call event's reply destination.

Destination to use when replying through, for example, the action {reply, From, Reply} to a process that has called the gen_statem server using call/2,3.

generic_timeout()

(not exported) (since OTP 20.0)

-type generic_timeout() :: Time :: timeout() | integer().

How long to wait for a named time-out event.

Starts a timer set by timeout_action/0 {{timeout, Name}, Time, EventContent [, Options]}.

When the timer expires an event of event_type/0 {timeout, Name} will be generated. See erlang:start_timer/4 for how Time and Options are interpreted. Future erlang:start_timer/4 Options will not necessarily be supported.

If Time is infinity, no timer is started, as it never would expire anyway.

If Time is relative and 0 no timer is actually started, instead the time-out event is enqueued to ensure that it gets processed before any not yet received external event.

Setting a timer with the same Name while it is running will restart it with the new time-out value. Therefore it is possible to cancel a specific time-out by setting it to infinity. It can also be cancelled more explicitly with the {{timeout, Name}, cancel} action.

The timer EventContent can be updated with the {{timeout, Name}, update, NewEventContent} action without affecting the time of expiry.

hibernate()

(not exported) (since OTP 19.0)

-type hibernate() :: boolean().

Hibernate the server process.

If true, hibernates the gen_statem by calling proc_lib:hibernate/3 before going into receive to wait for a new external event.

There is also a server start option {hibernate_after, Timeout} for automatic hibernation.

Note

If there are enqueued events to process when hibernation is requested, this is optimized by not hibernating but instead calling erlang:garbage_collect/0 to simulate, in a more efficient way, that the gen_statem entered hibernation and immediately got awakened by an enqueued event.

init_result(StateType)

(since OTP 19.0)

-type init_result(StateType) :: init_result(StateType, term()).

init_result(StateType, DataType)

(since OTP 19.0)

-type init_result(StateType, DataType) ::
          {ok, State :: StateType, Data :: DataType} |
          {ok, State :: StateType, Data :: DataType, Actions :: [action()] | action()} |
          ignore |
          {stop, Reason :: term()} |
          {error, Reason :: term()}.

The return value from Module:init/1.

For a succesful initialization, State is the initial state/0, and Data the initial server data/0 of the gen_statem.

The Actions are executed when entering the first state just as for a state callback, except that the action postpone is forced to false since there is no event to postpone.

For an unsuccesful initialization, {stop, Reason}, {error, Reason}, or ignore should be used; see start_link/3,4.

{error, Reason} has been allowed since OTP 26.0.

The {ok, ...} tuples have existed since OTP 19.1, before that they were not ok tagged. This was before gen_statem replaced gen_fsm in OTP 20.0.

postpone()

(not exported) (since OTP 19.0)

-type postpone() :: boolean().

Postpone an event to handle it later.

If true, postpones the current event. After a state change (NextState =/= State), it is retried.

reply_action()

(since OTP 19.0)

-type reply_action() :: {reply, From :: from(), Reply :: term()}.

Reply to a call/2,3.

This transition action can be invoked by returning it from the state callback, from Module:init/1 or by passing it to enter_loop/4,5,6.

It does not set any transition_option() but instead replies to a caller waiting for a reply in call/3. From must be the term from argument {call, From} in a call to a state callback.

Note that using this action from Module:init/1 or enter_loop/4,5,6 would be weird on the border of witchcraft since there has been no earlier call to a state callback in this server.

reply_tag()

(since OTP 19.0)

-opaque reply_tag()

A handle that associates a reply to the corresponding request.

request_id()

(since OTP 19.0)

-opaque request_id()

An opaque request identifier. See send_request/2 for details.

request_id_collection()

(since OTP 19.0)

-opaque request_id_collection()

An opaque collection of request identifiers (request_id/0).

Each request identifier can be associated with a label chosen by the user. For more information see reqids_new/0.

response_timeout()

(not exported) (since OTP 19.0)

-type response_timeout() :: timeout() | {abs, integer()}.

Response time-out for an asynchronous call.

Used to set a time limit on how long to wait for a response using either receive_response/2, receive_response/3, wait_response/2, or wait_response/3. The time unit used is millisecond.

Currently valid values:

0..4294967295 - Time-out relative to current time in milliseconds.

infinity - Infinite time-out. That is, the operation will never time out.

{abs, Timeout} - An absolute Erlang monotonic time time-out in milliseconds. That is, the operation will time out when erlang:monotonic_time(millisecond) returns a value larger than or equal to Timeout. Timeout is not allowed to identify a time further into the future than 4294967295 milliseconds. Specifying the time-out using an absolute value is especially handy when you have a deadline for responses corresponding to a complete collection of requests (request_id_collection/0), since you do not have to recalculate the relative time until the deadline over and over again.

server_name()

(since OTP 19.0)

-type server_name() ::
          {local, atom()} | {global, GlobalName :: term()} | {via, RegMod :: module(), Name :: term()}.

Server name specification: local, global, or via registered.

Name specification to use when starting a gen_statem server. See start_link/3 and server_ref/0 below.

server_ref()

(since OTP 19.0)

-type server_ref() ::
          pid() |
          (LocalName :: atom()) |
          {Name :: atom(), Node :: atom()} |
          {global, GlobalName :: term()} |
          {via, RegMod :: module(), ViaName :: term()}.

Server specification: pid/0 or registered server_name/0.

To be used in call/2,3 to specify the server.

It can be:

pid() | LocalName - The gen_statem is locally registered.

{Name, Node} - The gen_statem is locally registered on another node.

{global, GlobalName} - The gen_statem is globally registered in global.

{via, RegMod, ViaName} - The gen_statem is registered in an alternative process registry. The registry callback module RegMod is to export functions register_name/2, unregister_name/1, whereis_name/1, and send/2, which are to behave like the corresponding functions in global. Thus, {via, global, GlobalName} is the same as {global, GlobalName}.

start_mon_ret()

(since OTP 19.0)

-type start_mon_ret() :: {ok, {pid(), reference()}} | ignore | {error, term()}.

Return value from the start_monitor/3,4 functions.

As for start_link/4 but a succesful return wraps the process ID and the monitor reference in a {ok, {pid(),reference()}} tuple.

start_opt()

(since OTP 19.0)

-type start_opt() ::
          {timeout, Time :: timeout()} | {spawn_opt, [proc_lib:start_spawn_option()]} | enter_loop_opt().

Server start options for the start/3,4, start_link/3,4, and start_monitor/3,4 functions.

See start_link/4.

start_ret()

(since OTP 19.0)

-type start_ret() :: {ok, pid()} | ignore | {error, term()}.

Return value from the start/3,4 and start_link/3,4 functions.

See start_link/4.

state()

(not exported) (since OTP 19.0)

-type state() :: state_name() | term().

State name or state term.

If the callback mode is handle_event_function, the state can be any term. After a state change (NextState =/= State), all postponed events are retried.

Comparing two states for strict equality is assumed to be a fast operation, since for every state transition the gen_statem engine has to deduce if it is a state change.

Note

The smaller the state term, in general, the faster the comparison.

Note that if the "same" state term is returned for a state transition (or a return action without a NextState field is used), the comparison for equality is always fast because that can be seen from the term handle.

But if a newly constructed state term is returned, both the old and the new state terms will have to be traversed until an inequality is found, or until both terms have been fully traversed.

So it is possible to use large state terms that are fast to compare, but very easy to accidentally mess up. Using small state terms is the safe choice.

state_callback_result(ActionType, DataType)

(not exported) (since OTP 19.0)

-type state_callback_result(ActionType, DataType) ::
          {keep_state, NewData :: DataType} |
          {keep_state, NewData :: DataType, Actions :: [ActionType] | ActionType} |
          keep_state_and_data |
          {keep_state_and_data, Actions :: [ActionType] | ActionType} |
          {repeat_state, NewData :: DataType} |
          {repeat_state, NewData :: DataType, Actions :: [ActionType] | ActionType} |
          repeat_state_and_data |
          {repeat_state_and_data, Actions :: [ActionType] | ActionType} |
          stop |
          {stop, Reason :: term()} |
          {stop, Reason :: term(), NewData :: DataType} |
          {stop_and_reply, Reason :: term(), Replies :: [reply_action()] | reply_action()} |
          {stop_and_reply,
           Reason :: term(),
           Replies :: [reply_action()] | reply_action(),
           NewData :: DataType}.

Return value from any state callback.

ActionType is enter_action/0 if the state callback was called with a state enter call, and action/0 if the state callback was called with an event.

{keep_state, NewData [, Actions]} - The same as {next_state, CurrentState, NewData [, Actions]}.

keep_state_and_data | {keep_state_and_data, Actions} - The same as {keep_state, CurrentData [, Actions]}.

{repeat_state, NewData [, Actions]} - If the gen_statem runs with state enter calls, the state enter call is repeated, see type transition_option/0. Other than that {repeat_state, NewData [, Actions]} is the same as {keep_state, NewData [, Actions]}.

repeat_state_and_data | {repeat_state_and_data, Actions} - The same as {repeat_state, CurrentData [, Actions]}.

{stop, Reason [, NewData]} - Terminates the gen_statem by calling Module:terminate/3 with Reason and NewData, if specified. An exit signal with this reason is sent to linked processes and ports.

stop - The same as {stop, normal}.

{stop_and_reply, Reason, Replies [, NewData]} - Sends all Replies, then terminates the gen_statem like with {stop, Reason [, NewData]}.

All these terms are tuples or atoms and will be so in all future versions of gen_statem.

state_enter()

(not exported) (since OTP 19.2)

-type state_enter() :: state_enter.

Callback mode modifier for state enter calls: the atom state_enter.

Both callback modes can use state enter calls, and this is selected by adding this state_enter flag to the callback mode return value from Module:callback_mode/0.

If Module:callback_mode/0 returns a list containing state_enter, the gen_statem engine will, at every state change, that is; NextState =/= CurrentState, call the state callback with arguments (enter, OldState, Data) or (enter, OldState, State, Data), depending on the callback mode.

This may look like an event but is really a call performed after the previous state callback returned, and before any event is delivered to the new state callback. See Module:StateName/3 and Module:handle_event/4. A state enter call may be repeated without doing a state change by returning a repeat_state or repeat_state_and_data action from the state callback.

If Module:callback_mode/0 does not return a list containing state_enter, no state enter calls are done.

If Module:code_change/4 should transform the state, it is regarded as a state rename and not a state change, which will not cause a state enter call.

Note that a state enter call will be done right before entering the initial state, which may be seen as a state change from no state to the initial state. In this case OldState =:= State, which cannot happen for a subsequent state change, but will happen when repeating the state enter call.

state_enter_result(State)

(since OTP 19.0)

-type state_enter_result(State) :: state_enter_result(State, term()).

state_enter_result(State, DataType)

(since OTP 19.0)

-type state_enter_result(State, DataType) ::
          {next_state, State, NewData :: DataType} |
          {next_state, State, NewData :: DataType, Actions :: [enter_action()] | enter_action()} |
          state_callback_result(enter_action(), DataType).

Return value from a state callback after a state enter call.

State is the current state and it cannot be changed since the state callback was called with a state enter call.

{next_state, State, NewData [, Actions]} - The gen_statem does a state transition to State, which has to be equal to the current state, sets NewData, and executes all Actions.

state_name()

(not exported) (since OTP 19.0)

-type state_name() :: atom().

State name in callback mode state_functions.

If the callback mode is state_functions, the state must be an atom. After a state change (NextState =/= State), all postponed events are retried. Note that the state terminate is not possible to use since it would collide with the optional callback function Module:terminate/3.

state_timeout()

(not exported) (since OTP 19.3)

-type state_timeout() :: Time :: timeout() | integer().

How long to wait in the current state.

Starts a timer set by timeout_action/0, or {state_timeout, Time, EventContent [, Options]}.

When the timer expires an event of event_type/0 state_timeout will be generated. See erlang:start_timer/4 for how Time and Options are interpreted. Future erlang:start_timer/4 Options will not necessarily be supported.

A state change cancels this timer, if it is running. That is, if the timeout_action/0 that starts this timer is part of a list of action/0s for a state change, NextState =/= CurrentState, the timer runs in the NextState.

If the state machine stays in that new state, now the current state, the timer will run until it expires, which creates the time-out event. If the state machine changes states from the now current state, the timer is cancelled. During the state change from the now current state, a new state time-out may be started for the next NextState.

If the timeout_action/0 that starts this timer is part of a list of action/0s for a state transition that is not a state change, the timer runs in the current state.

If Time is infinity, no timer is started, as it never would expire anyway.

If Time is relative and 0 no timer is actually started, instead the the time-out event is enqueued to ensure that it gets processed before any not yet received external event.

Setting this timer while it is running will restart it with the new time-out value. Therefore it is possible to cancel this time-out by setting it to infinity. It can also be cancelled more explicitly with {state_timeout, cancel}.

The timer EventContent can be updated with the {state_timeout, update, NewEventContent} action without affecting the time of expiry.

timeout_action()

(not exported) (since OTP 19.0)

-type timeout_action() ::
          (Time :: event_timeout()) |
          {timeout, Time :: event_timeout(), EventContent :: event_content()} |
          {timeout,
           Time :: event_timeout(),
           EventContent :: event_content(),
           Options :: timeout_option() | [timeout_option()]} |
          {{timeout, Name :: term()}, Time :: generic_timeout(), EventContent :: event_content()} |
          {{timeout, Name :: term()},
           Time :: generic_timeout(),
           EventContent :: event_content(),
           Options :: timeout_option() | [timeout_option()]} |
          {state_timeout, Time :: state_timeout(), EventContent :: event_content()} |
          {state_timeout,
           Time :: state_timeout(),
           EventContent :: event_content(),
           Options :: timeout_option() | [timeout_option()]} |
          timeout_cancel_action() |
          timeout_update_action().

Event time-out, generic time-outs or state time-out.

These transition actions can be invoked by returning them from the state callback, from Module:init/1 or by passing them to enter_loop/4,5,6.

These time-out actions sets time-out transition options.

Time - Short for {timeout, Time, Time}, that is, the time-out message is the time-out time. This form exists to allow the state callback return value {next_state, NextState, NewData, Time} like in gen_fsm.

{timeout, Time, EventContent [, Options]} - Sets the transition_option/0 event_timeout/0 to Time with EventContent, and time-out options Options.

{{timeout,Name}, Time, EventContent [, Options]} - Sets the transition_option/0 generic_timeout/0 to Time for time-out Name with EventContent, and time-out options Options.
Since OTP 20.0.

{state_timeout, Time, EventContent [, Options]} - Sets the transition_option/0 state_timeout/0 to Time with EventContent, and time-out options Options.
Since OTP 19.3.

timeout_cancel_action()

(not exported) (since OTP 22.1)

-type timeout_cancel_action() ::
          {timeout, cancel} | {{timeout, Name :: term()}, cancel} | {state_timeout, cancel}.

Clearer way to cancel a time-out than the original setting it to 'infinity'.

It has always been possible to cancel a time-out using timeout_action/0 with Time = infinity, since setting a new time-out time overrides a running timer, and since setting the time to infinity is optimized to not setting a timer (that never will expire). Using this action shows the intention more clearly.

timeout_event_type()

(not exported) (since OTP 19.0)

-type timeout_event_type() :: timeout | {timeout, Name :: term()} | state_timeout.

Event time-out, generic time-out, or state time-out.

The time-out event types that the state machine can generate for itself with the corresponding timeout_action/0s:

Time-out type | Action | Event type
Event time-out | {timeout, Time, ...} | timeout
Generic time-out | {{timeout, Name}, Time, ...} | {timeout, Name}
State time-out | {state_timeout, Time, ...} | state_timeout

In short; the action to set a time-out with EventType is {EventType, Time, ...}.

timeout_option()

(not exported) (since OTP 19.0)

-type timeout_option() :: {abs, Abs :: boolean()}.

Time-out timer start option, to select absolute time of expiry.

If Abs is true an absolute timer is started, and if it is false a relative, which is the default. See erlang:start_timer/4 for details.

timeout_update_action()

(not exported) (since OTP 22.1)

-type timeout_update_action() ::
          {timeout, update, EventContent :: event_content()} |
          {{timeout, Name :: term()}, update, EventContent :: event_content()} |
          {state_timeout, update, EventContent :: event_content()}.

Update the EventContent without affecting the time of expiry.

Sets a new EventContent for a running time-out timer. See timeout_action() for how to start a time-out.

If no time-out of this type is active, instead inserts the time-out event just like when starting a time-out with relative Time = 0. This is a time-out autostart with immediate expiry, so there will be noise for example if a generic time-out name was misspelled.

transition_option()

(since OTP 19.0)

-type transition_option() ::
          postpone() | hibernate() | event_timeout() | generic_timeout() | state_timeout().

State transition options set by actions.

These determine what happens during the state transition. The state transition takes place when the state callback has processed an event and returns. Here are the sequence of steps for a state transition:

All returned actions are processed in order of appearance. In this step all replies generated by any reply_action/0 are sent. Other actions set transition_option/0s that come into play in subsequent steps.

If state enter calls are used, it is either the initial state or one of the callback results repeat_state or repeat_state_and_data is used the gen_statem engine calls the current state callback with arguments (enter, State, Data) or (enter, State, State, Data) (depending on callback mode) and when it returns starts again from the top of this sequence.

If state enter calls are used, and the state changes, the gen_statem engine calls the new state callback with arguments (enter, OldState, Data) or (enter, OldState, State, Data) (depending on callback mode) and when it returns starts again from the top of this sequence.

If postpone/0 is true, the current event is postponed.

If this is a state change, the queue of incoming events is reset to start with the oldest postponed.

All events stored with action/0 next_event are inserted to be processed before previously queued events.

Time-out timers event_timeout/0, generic_timeout/0 and state_timeout/0 are handled. Time-outs with zero time are guaranteed to be delivered to the state machine before any external not yet received event so if there is such a time-out requested, the corresponding time-out zero event is enqueued as the newest received event; that is after already queued events such as inserted and postponed events.

Any event cancels an event_timeout/0 so a zero time event time-out is only generated if the event queue is empty.

A state change cancels a state_timeout/0 and any new transition option of this type belongs to the new state, that is; a state_timeout/0 applies to the state the state machine enters.

If there are enqueued events the state callback for the possibly new state is called with the oldest enqueued event, and we start again from the top of this sequence.

Otherwise the gen_statem goes into receive or hibernation (if hibernate/0 is true) to wait for the next message. In hibernation the next non-system event awakens the gen_statem, or rather the next incoming message awakens the gen_statem, but if it is a system event it goes right back into hibernation. When a new message arrives the state callback is called with the corresponding event, and we start again from the top of this sequence.

Note

The behaviour of a time-out zero (a time-out with time 0) differs subtly from Erlang's receive ... after 0 ... end.

The latter receives one message if there is one, while using the timeout_action/0 {timeout, 0} does not receive any external event.

gen_server's time-out works like Erlang's receive ... after 0 ... end, in contrast to gen_statem.

Callbacks

callback_mode()

(since OTP 19.1)

-callback callback_mode() -> callback_mode_result().

Select the callback mode and possibly state enter calls.

This function is called by a gen_statem when it needs to find out the callback mode of the callback module.

The value is cached by gen_statem for efficiency reasons, so this function is only called once after server start, after code change, and after changing the callback module, but before the first state callback in the current callback module's code is called. More occasions may be added in future versions of gen_statem.

Server start happens either when Module:init/1 returns or when enter_loop/4,5,6 is called. Code change happens when Module:code_change/4 returns. A change of the callback module happens when a state callback returns any of the actions change_callback_module, push_callback_module or pop_callback_module.

The CallbackMode is either just callback_mode/0 or a list containing callback_mode/0 and possibly the atom state_enter.

Note

If this function's body does not return an inline constant value the callback module is doing something strange.

code_change(OldVsn, OldState, OldData, Extra)

(since OTP 19.0) (optional)

-callback code_change(OldVsn :: term() | {down, term()},
                      OldState :: state(),
                      OldData :: data(),
                      Extra :: term()) ->
                         {ok, NewState :: state(), NewData :: data()} | (Reason :: term()).

Update the state and data after code change.

This function is called by a gen_statem when it is to update its internal state during a release upgrade/downgrade, that is, when the instruction {update, Module, Change, ...}, where Change = {advanced, Extra}, is specified in the appup file. For more information, see Release Handling Instructions in OTP Design Principles.

For an upgrade, OldVsn is Vsn, and for a downgrade, OldVsn is {down, Vsn}. Vsn is defined by the vsn attribute(s) of the old version of the callback module Module. If no such attribute is defined, the version is the checksum of the Beam file.

OldState and OldData is the internal state of the gen_statem.

Extra is passed "as is" from the {advanced, Extra} part of the update instruction.

If successful, the function must return the updated internal state in an {ok, NewState, NewData} tuple.

If the function returns a failure Reason, the ongoing upgrade fails and rolls back to the old release. Note that Reason cannot be an {ok, _, _} tuple since that will be regarded as a {ok, NewState, NewData} tuple, and that a tuple matching {ok, _} is an also invalid failure Reason. It is recommended to use an atom as Reason since it will be wrapped in an {error, Reason} tuple.

Also note when upgrading a gen_statem, this function and hence the Change = {advanced, Extra} parameter in the appup file is not only needed to update the internal state or to act on the Extra argument. It is also needed if an upgrade or downgrade should change callback mode, or else the callback mode after the code change will not be honoured, most probably causing a server crash.

If the server changes callback module using any of the actions change_callback_module, push_callback_module, or pop_callback_module, be aware that it is always the current callback module that will get this callback call. That the current callback module handles the current state and data update should be no surprise, but it must be able to handle even parts of the state and data that it is not familiar with, somehow.

In the supervisor child specification there is a list of modules which is recommended to contain only the callback module. For a gen_statem with multiple callback modules there is no real need to list all of them, it may not even be possible since the list could change after code upgrade. If this list would contain only the start callback module, as recommended, what is important is to upgrade that module whenever a synchronized code replacement is done. Then the release handler concludes that an upgrade that upgrades that module needs to suspend, code change, and resume any server whose child specification declares that it is using that module. And again; the current callback module will get the Module:code_change/4 call.

Note

If a release upgrade/downgrade with Change = {advanced, Extra} specified in the .appup file is made when Module:code_change/4 is not implemented the process will crash with exit reason undef.

format_status(Status)

(since OTP 25.0) (optional)

-callback format_status(Status) -> NewStatus when Status :: format_status(), NewStatus :: format_status().

Format/limit the status value.

This function is called by a gen_statem process in order to format/limit the server status for debugging and logging purposes.

It is called in the following situations:

sys:get_status/1,2 is invoked to get the gen_statem status.

The gen_statem process terminates abnormally and logs an error.

This function is useful for changing the form and appearance of the gen_statem status for these cases. A callback module wishing to change the sys:get_status/1,2 return value and how its status appears in termination error logs, exports an instance of Module:format_status/1, which will get a map Status that describes the current state of the gen_statem, and shall return a map NewStatus containing the same keys as the input map, but it may transform some values.

One use case for this function is to return compact alternative state representations to avoid having large state terms printed in log files. Another is to hide sensitive data from being written to the error log.

Example:

format_status(Status) ->
  maps:map(
    fun(state,State) ->
            maps:remove(private_key, State);
       (message,{password, _Pass}) ->
            {password, removed};
       (_,Value) ->
            Value
    end, Status).

Note

This callback is optional, so a callback module does not need to export it. The gen_statem module provides a default implementation of this function that returns {State, Data}.

If this callback is exported but fails, to hide possibly sensitive data, the default function will instead return {State, Info}, where Info says nothing but the fact that Module:format_status/2 has crashed.

format_status/2

(since OTP 19.0) (optional)

This callback is deprecated. the callback gen_statem:format_status(_,_) is deprecated; use format_status/1 instead.

-callback format_status(StatusOption, [[{Key :: term(), Value :: term()}] | state() | data()]) ->
                           Status :: term()
                           when StatusOption :: normal | terminate.

Format/limit the status value.

This function is called by a gen_statem process in in order to format/limit the server state for debugging and logging purposes.

It is called in the following situations:

One of sys:get_status/1,2 is invoked to get the gen_statem status. Opt is set to the atom normal for this case.

The gen_statem terminates abnormally and logs an error. Opt is set to the atom terminate for this case.

This function is useful for changing the form and appearance of the gen_statem status for these cases. A callback module wishing to change the sys:get_status/1,2 return value and how its status appears in termination error logs, should export an instance of Module:format_status/2, that returns a term describing the current status of the gen_statem.

PDict is the current value of the process dictionary of the gen_statem.

State is the internal state of the gen_statem.

Data is the internal server data of the gen_statem.

The function is to return Status, a term that contains the appropriate details of the current state and status of the gen_statem. There are no restrictions of the form Status can take, but for the sys:get_status/1,2 case (when Opt is normal), the recommended form for the Status value is [{data, [{"State", Term}]}], where Term provides relevant details of the gen_statem state. Following this recommendation is not required, but it makes the callback module status consistent with the rest of the sys:get_status/1,2 return value.

One use for this function is to return compact alternative state representations to avoid having large state terms printed in log files. Another use is to hide sensitive data from being written to the error log.

Note

This callback is optional, so a callback module does not need to export it. The gen_statem module provides a default implementation of this function that returns {State, Data}.

If this callback is exported but fails, to hide possibly sensitive data, the default function will instead return {State, Info}, where Info says nothing but the fact that Module:format_status/2 has crashed.

handle_event/4

(since OTP 19.0) (optional)

-callback handle_event(enter, OldState, CurrentState, Data) -> state_enter_result(CurrentState)
                          when OldState :: state(), CurrentState :: state(), Data :: data();
                      (EventType, EventContent, CurrentState, Data) -> event_handler_result(state())
                          when
                              EventType :: event_type(),
                              EventContent :: event_content(),
                              CurrentState :: state(),
                              Data :: data().

State callback in callback mode handle_event_function.

Whenever a gen_statem receives an event from call/2,3, cast/2, or as a normal process message, this function is called.

If EventType is {call, From}, the caller waits for a reply. The reply can be sent from this or from any other state callback by returning with {reply, From, Reply} in Actions, in Replies, or by calling reply(From, Reply).

If this function returns with a next state that does not match equal (=/=) to the current state, all postponed events are retried in the next state.

For options that can be set and actions that can be done by gen_statem after returning from this function, see action/0.

When the gen_statem runs with state enter calls, this function is also called with arguments (enter, OldState, ...) during every state change. In this case there are some restrictions on the actions that may be returned:

postpone/0 is not allowed since a state enter call is not an event so there is no event to postpone.

{next_event, _, _} is not allowed since using state enter calls should not affect how events are consumed and produced.

It is not allowed to change states from this call. Should you return {next_state, NextState, ...} with NextState =/= State the gen_statem crashes.

Note that it is actually allowed to use {repeat_state, NewData, ...} although it makes little sense since you immediately will be called again with a new state enter call making this just a weird way of looping, and there are better ways to loop in Erlang.

If you do not update NewData and have some loop termination condition, or if you use {repeat_state_and_data, _} or repeat_state_and_data you have an infinite loop!

You are advised to use {keep_state, ...}, {keep_state_and_data, _} or keep_state_and_data since changing states from a state enter call is not possible anyway.

Note the fact that you can use throw to return the result, which can be useful. For example to bail out with throw(keep_state_and_data) from deep within complex code that cannot return {next_state, State, Data} because State or Data is no longer in scope.

init(Args)

(since OTP 19.0)

-callback init(Args :: term()) -> init_result(state()).

Initialize the state machine.

Whenever a gen_statem is started using start_link/3,4, start_monitor/3,4, or start/3,4, this function is called by the new process to initialize the implementation state and server data.

Args is the Args argument provided to that start function.

Note

Note that if the gen_statem is started through proc_lib and enter_loop/4,5,6, this callback will never be called. Since this callback is not optional it can in that case be implemented as:

-spec init(_) -> no_return().
init(Args) -> erlang:error(not_implemented, [Args]).

'StateName'/3

(since OTP 19.0) (optional)

-callback 'StateName'(enter, OldStateName :: state_name(), data()) -> state_enter_result(state_name);
                     (EventType :: event_type(), EventContent :: event_content(), Data :: data()) ->
                         event_handler_result(state_name()).

State callback in callback mode state_functions.

State callback that handles all events in state StateName, where StateName :: state_name() has to be an atom/0.

StateName cannot be terminate since that would collide with the callback function Module:terminate/3.

Besides that when doing a state change the next state always has to be an atom/0, this function is equivalent to Module:handle_event(​EventType, EventContent, ?FUNCTION_NAME, Data), which is the state callback in callback mode handle_event_function.

terminate/3

(since OTP 19.0) (optional)

-callback terminate(Reason :: normal | shutdown | {shutdown, term()} | term(),
                    CurrentState :: state(),
                    data()) ->
                       any().

Handle state machine termination.

This function is called by a gen_statem when it is about to terminate. It is to be the opposite of Module:init/1 and do any necessary cleaning up. When it returns, the gen_statem terminates with Reason. The return value is ignored.

Reason is a term denoting the stop reason and State is the internal state of the gen_statem.

Reason depends on why the gen_statem is terminating. If it is because another callback function has returned, a stop tuple {stop, Reason} in Actions, Reason has the value specified in that tuple. If it is because of a failure, Reason is the error reason.

If the gen_statem is part of a supervision tree and is ordered by its supervisor to terminate, this function is called with Reason = shutdown if both the following conditions apply:

The gen_statem process has been set to trap exit signals.

The shutdown strategy as defined in the supervisor's child specification is an integer time-out value, not brutal_kill.

Even if the gen_statem is not part of a supervision tree, this function is called if it receives an 'EXIT' message from its parent. Reason is the same as in the 'EXIT' message.

If the gen_statem process is not set up to trap exit signals it is immediately terminated, just like any process, and this function is not called.

Notice that for any other reason than normal, shutdown, or {shutdown, Term}, the gen_statem is assumed to terminate because of an error and an error report is issued using logger.

When the gen_statem process exits, an exit signal with the same reason is sent to linked processes and ports, just as for any process.

Functions

call(ServerRef, Request)

(since OTP 19.0)

-spec call(ServerRef :: server_ref(), Request :: term()) -> Reply :: term().

Equivalent to call(ServerRef, Request, infinity).

call(ServerRef, Request, Timeout)

(since OTP 19.0)

-spec call(ServerRef :: server_ref(),
           Request :: term(),
           Timeout :: timeout() | {clean_timeout, T :: timeout()} | {dirty_timeout, T :: timeout()}) ->
              Reply :: term().

Call a server: send request and wait for response.

Makes a synchronous call to the gen_statem ServerRef by sending a request and waiting until the response arrives.

The gen_statem calls the state callback with event_type/0 {call, From} and event content Request.

The server's reply is sent from a state callback, by returning a transition action {reply, From, Reply}, calling reply(Replies) with such a reply action in the Replies list, or calling reply(From, Reply).

Timeout is an integer > 0, which specifies how many milliseconds to wait for a reply, or the atom infinity to wait indefinitely, which is the default. If no reply is received within the specified time, the function call fails.

Previous issue with late replies that could occur when having network issues or using dirty_timeout is now prevented by use of process aliases. {clean_timeout, T} and {dirty_timeout, T} therefore no longer serves any purpose and will work the same as Timeout while all of them also being equally efficient.

The call can also fail, for example, if the gen_statem dies before or during this function call.

When this call fails it exits the calling process. The exit term is on the form {Reason, Location} where Location = {gen_statem, call, ArgList}. See gen_server:call/3 that has a description of relevant values for the Reason in the exit term.

cast(ServerRef, Msg)

(since OTP 19.0)

-spec cast(ServerRef :: server_ref(), Msg :: term()) -> ok.

Cast an event to a server.

Sends an asynchronous cast event to the gen_statem ServerRef and returns ok immediately, ignoring if the destination node or gen_statem does not exist.

The gen_statem calls the state callback with event_type/0 cast and event content Msg.

check_response(Msg, ReqId)

(since OTP 23.0)

-spec check_response(Msg, ReqId) -> Result
                        when
                            Msg :: term(),
                            ReqId :: request_id(),
                            Response ::
                                {reply, Reply :: term()} | {error, {Reason :: term(), server_ref()}},
                            Result :: Response | no_reply.

Check if a received message is a request response.

Checks if Msg is a response corresponding to the request identifier ReqId. The request must have been made by send_request/2 and by the same process calling this function.

If Msg is a reply to the handle ReqId the result of the request is returned in Reply. Otherwise this function returns no_reply and no cleanup is done, and thus the function shall be invoked repeatedly until the response is returned.

See call/3 about how the request is handled and the Reply is sent by the gen_statem server.

If the gen_statem server process has died when this function is called, that is; Msg reports the server's death, this function returns an error return with the exit Reason.

check_response(Msg, ReqIdCollection, Delete)

(since OTP 25.0)

-spec check_response(Msg, ReqIdCollection, Delete) -> Result
                        when
                            Msg :: term(),
                            ReqIdCollection :: request_id_collection(),
                            Delete :: boolean(),
                            Response ::
                                {reply, Reply :: term()} | {error, {Reason :: term(), server_ref()}},
                            Result ::
                                {Response,
                                 Label :: term(),
                                 NewReqIdCollection :: request_id_collection()} |
                                no_request | no_reply.

Check if a received message is a request response in a collection.

Check if Msg is a response corresponding to a request identifier stored in ReqIdCollection. All request identifiers of ReqIdCollection must correspond to requests that have been made using send_request/2 or send_request/4, by the process calling this function.

The Label in the response equals the Label associated with the request identifier that the response corresponds to. The Label of a request identifier is associated when storing the request id in a collection, or when sending the request using send_request/4.

Compared to check_response/2, the returned result or exception associated with a specific request identifier will be wrapped in a 3-tuple {Response, Label, NewReqIdCollection}. Response is the value that would have been produced by check_response/2, Label is the value associated with the specific request identifier and NewReqIdCollection is a possibly modified request identifier collection.

If ReqIdCollection is empty, no_request is returned.

If Msg does not correspond to any of the request identifiers in ReqIdCollection, no_reply is returned.

If Delete equals true, the association with Label has been deleted from ReqIdCollection in the resulting NewReqIdCollection. If Delete is false, NewReqIdCollection will equal ReqIdCollection. Note that deleting an association is not for free and that a collection containing already handled requests can still be used by subsequent calls to wait_response/3, check_response/3, and receive_response/3.

However, without deleting handled associations, the above calls will not be able to detect when there are no more outstanding requests to handle, so you will have to keep track of this some other way than relying on a no_request return. Note that if you pass a collection only containing associations of already handled or abandoned requests to this function, it will always return no_reply.

enter_loop(Module, Opts, State, Data)

(since OTP 19.1)

-spec enter_loop(Module :: term(), Opts :: term(), State :: term(), Data :: term()) -> no_return().

Equivalent to enter_loop(Module, Opts, State, Data, self(), []).

enter_loop(Module, Opts, State, Data, Server_or_Actions)

(since OTP 19.0)

-spec enter_loop(Module :: term(), Opts :: term(), State :: term(), Data :: term(), Actions) ->
                    no_return()
                    when Actions :: list();
                (Module :: term(), Opts :: term(), State :: term(), Data :: term(), Server) ->
                    no_return()
                    when Server :: server_name() | pid().

Make the calling process become a gen_statem server.

With argument Actions, equivalent to enter_loop(Module, Opts, State, Data, self(), Actions).

Otherwise equivalent to enter_loop(Module, Opts, State, Data, Server, []).

enter_loop(Module, Opts, State, Data, Server, Actions)

(since OTP 19.0)

-spec enter_loop(Module :: module(),
                 Opts :: [enter_loop_opt()],
                 State :: state(),
                 Data :: data(),
                 Server :: server_name() | pid(),
                 Actions :: [action()] | action()) ->
                    no_return().

Make the calling process become a gen_statem server.

Does not return, instead the calling process enters the gen_statem receive loop and becomes a gen_statem server. The process must have been started using one of the start functions in proc_lib. The user is responsible for any initialization of the process, including registering a name for it.

This function is useful when a more complex initialization procedure is needed than the gen_statem Module:init/1 callback offers.

Module and Opts have the same meanings as when calling start[link | monitor]/3,4.

If Server is self/0 an anonymous server is created just as when using start[link |_monitor]/3. If Server is a server_name/0 a named server is created just as when using start[link |_monitor]/4. However, the server_name/0 name must have been registered accordingly before this function is called.

State, Data, and Actions have the same meanings as in the return value of Module:init/1. Also, the callback module does not need to export a Module:init/1 function.

The function fails if the calling process was not started by a proc_lib start function, or if it is not registered according to server_name/0.

receive_response(ReqId)

(since OTP 24.0)

-spec receive_response(ReqId) -> Result
                          when
                              ReqId :: request_id(),
                              Response ::
                                  {reply, Reply :: term()} | {error, {Reason :: term(), server_ref()}},
                              Result :: Response | timeout.

Equivalent to receive_response(ReqId, infinity).

receive_response(ReqId, Timeout)

(since OTP 24.0)

-spec receive_response(ReqId, Timeout) -> Result
                          when
                              ReqId :: request_id(),
                              Timeout :: response_timeout(),
                              Response ::
                                  {reply, Reply :: term()} | {error, {Reason :: term(), server_ref()}},
                              Result :: Response | timeout.

Receive a request response.

Receive a response corresponding to the request identifier ReqId. The request must have been made by send_request/2 to the gen_statem process. This function must be called from the same process from which send_request/2 was made.

Timeout specifies how long to wait for a response. If no response is received within the specified time, this function returns timeout. Assuming that the server executes on a node supporting aliases (introduced in OTP 24) the request will also be abandoned. That is, no response will be received after a time-out. Otherwise, a stray response might be received at a later time.

See call/3 about how the request is handled and the Reply is sent by the gen_statem server.

If the gen_statem server process is dead or dies while this function waits for the reply, it returns an error return with the exit Reason.

The difference between wait_response/2 and receive_response/2 is that receive_response/2 abandons the request at time-out so that a potential future response is ignored, while wait_response/2 does not.

receive_response(ReqIdCollection, Timeout, Delete)

(since OTP 25.0)

-spec receive_response(ReqIdCollection, Timeout, Delete) -> Result
                          when
                              ReqIdCollection :: request_id_collection(),
                              Timeout :: response_timeout(),
                              Delete :: boolean(),
                              Response ::
                                  {reply, Reply :: term()} | {error, {Reason :: term(), server_ref()}},
                              Result ::
                                  {Response,
                                   Label :: term(),
                                   NewReqIdCollection :: request_id_collection()} |
                                  no_request | timeout.

Receive a request response in a collection.

Receive a response in ReqIdCollection. All request identifiers of ReqIdCollection must correspond to requests that have been made using send_request/2 or send_request/4, and all requests must have been made by the process calling this function.

The Label in the response is the Label associated with the request identifier that the response corresponds to. The Label of a request identifier is associated when adding the request id to a collection, or when sending the request using send_request/4.

Compared to receive_response/2, the returned result or exception associated with a specific request identifier will be wrapped in a 3-tuple {Response, Label, NewReqIdCollection}. Response is the value that would have been produced by receive_response/2, Label is the value associated with the specific request identifier and NewReqIdCollection is a possibly modified request identifier collection.

If ReqIdCollection is empty, no_request will be returned.

Timeout specifies how long to wait for a response. If no response is received within the specified time, the function returns timeout. Assuming that the server executes on a node supporting aliases (introduced in OTP 24) all requests identified by ReqIdCollection will also be abandoned. That is, no responses will be received after a time-out. Otherwise, stray responses might be received at a later time.

The difference between receive_response/3 and wait_response/3 is that receive_response/3 abandons requests at time-out so that potential future responses are ignored, while wait_response/3 does not.

If Delete is true, the association with Label is deleted from ReqIdCollection in the resulting NewReqIdCollection. If Delete is false, NewReqIdCollection will equalReqIdCollection. Note that deleting an association is not for free and that a collection containing already handled requests can still be used by subsequent calls to wait_response/3, check_response/3, and receive_response/3.

However, without deleting handled associations, the above calls will not be able to detect when there are no more outstanding requests to handle, so you will have to keep track of this some other way than relying on a no_request return. Note that if you pass a collection only containing associations of already handled or abandoned requests to this function, it will always block until Timeout expires and then return timeout.

reply(Replies)

(since OTP 19.0)

-spec reply(Replies :: [reply_action()] | reply_action()) -> ok.

Send one or multiple call replies.

This funcion can be used by a gen_statem callback to explicitly send one or multiple replies to processes waiting for call requests' replies, when it is impractical or impossible to return reply_action/0s from a state callback.

Note

A reply sent with this function is not visible in sys debug output.

reply(From, Reply)

(since OTP 19.0)

-spec reply(From :: from(), Reply :: term()) -> ok.

Send a call Reply to From.

This funcion can be used by a gen_statem callback to explicitly send a reply to a process waiting for a call requests' reply, when it is impractical or impossible to return a reply_action/0 from a state callback.

Note

A reply sent with this function is not visible in sys debug output.

reqids_add(ReqId, Label, ReqIdCollection)

(since OTP 25.0)

-spec reqids_add(ReqId :: request_id(), Label :: term(), ReqIdCollection :: request_id_collection()) ->
                    NewReqIdCollection :: request_id_collection().

Store a request identifier in a colletion.

Stores ReqId and associates a Label with the request identifier by adding this information to ReqIdCollection and returning the resulting request identifier collection.

reqids_new()

(since OTP 25.0)

-spec reqids_new() -> NewReqIdCollection :: request_id_collection().

Create an empty request identifier collection.

Returns a new empty request identifier collection. A request identifier collection can be used to handle multiple outstanding requests.

Request identifiers of requests made by send_request/2 can be stored in a collection using reqids_add/3. Such a collection of request identifiers can later be used in order to get one response corresponding to a request in the collection by passing the collection as argument to receive_response/3, wait_response/3, or, check_response/3.

reqids_size/1 can be used to determine the number of request identifiers in a collection.

reqids_size(ReqIdCollection)

(since OTP 25.0)

-spec reqids_size(ReqIdCollection :: request_id_collection()) -> non_neg_integer().

Return the number of request identifiers in ReqIdCollection.

reqids_to_list(ReqIdCollection)

(since OTP 25.0)

-spec reqids_to_list(ReqIdCollection :: request_id_collection()) ->
                        [{ReqId :: request_id(), Label :: term()}].

Convert a request identifier collection to a list.

Returns a list of {ReqId, Label} tuples which corresponds to all request identifiers with their associated labels in ReqIdCollection.

send_request(ServerRef, Request)

(since OTP 23.0)

-spec send_request(ServerRef :: server_ref(), Request :: term()) -> ReqId :: request_id().

Send an asynchronous call request.

Sends Request to the gen_statem process identified by ServerRef and returns a request identifier ReqId.

The return value ReqId shall later be used with receive_response/2, wait_response/2, or check_response/2 to fetch the actual result of the request. Besides passing the request identifier directly to these functions, it can also be stored in a request identifier collection using reqids_add/3. Such a collection of request identifiers can later be used in order to get one response corresponding to a request in the collection by passing the collection as argument to receive_response/3, wait_response/3, or check_response/3. If you are about to store the request identifier in a collection, you may want to consider using send_request/4 instead.

The call gen_statem:wait_response(gen_statem:send_request(ServerRef, Request), Timeout) can be seen as equivalent to gen_statem:call(Server, Request, Timeout), ignoring the error handling.

See call/3 about how the request is handled and the Reply is sent by the gen_statem server.

The server's Reply is returned by one of the receive_response/1,2, wait_response/1,2, or check_response/2 functions.

send_request(ServerRef, Request, Label, ReqIdCollection)

(since OTP 25.0)

-spec send_request(ServerRef :: server_ref(),
                   Request :: term(),
                   Label :: term(),
                   ReqIdCollection :: request_id_collection()) ->
                      NewReqIdCollection :: request_id_collection().

Send an asynchronous call request and add it to a request identifier collection.

Sends Request to the gen_statem process identified by ServerRef. The Label will be associated with the request identifier of the operation and added to the returned request identifier collection NewReqIdCollection. The collection can later be used in order to get one response corresponding to a request in the collection by passing the collection as argument to receive_response/3, wait_response/3, or check_response/3.

The same as calling reqids_add(​send_request(ServerRef, Request),Label, ReqIdCollection), but slightly more efficient.

start(Module, Args, Opts)

(since OTP 19.0)

-spec start(Module :: module(), Args :: term(), Opts :: [start_opt()]) -> start_ret().

Start a server, neither linked nor registered.

Equivalent to start/4 except that the gen_statem process is not registered with any name service.

start(ServerName, Module, Args, Opts)

(since OTP 19.0)

-spec start(ServerName :: server_name(), Module :: module(), Args :: term(), Opts :: [start_opt()]) ->
               start_ret().

Start a server, registered but not linked.

Creates a standalone gen_statem process according to OTP design principles (using proc_lib primitives). As it does not get linked to the calling process, this start function cannot be used by a supervisor to start a child.

For a description of arguments and return values, see start_link/4.

start_link(Module, Args, Opts)

(since OTP 19.0)

-spec start_link(Module :: module(), Args :: term(), Opts :: [start_opt()]) -> start_ret().

Start a server, linked but not registered.

Equivalent to start_link/4 except that the gen_statem process is not registered with any name service.

start_link(ServerName, Module, Args, Opts)

(since OTP 19.0)

-spec start_link(ServerName :: server_name(), Module :: module(), Args :: term(), Opts :: [start_opt()]) ->
                    start_ret().

Start a server, linked and registered.

Creates a gen_statem process according to OTP design principles (using proc_lib primitives) that is spawned linked to the calling process. This is essential when the gen_statem must be part of a supervision tree so it gets linked to its supervisor.

The spawned gen_statem process calls Module:init/1 to initialize the server. To ensure a synchronized startup procedure, start_link/3,4 does not return until Module:init/1 has returned or failed.

ServerName specifies the server_name/0 to register for the gen_statem process. If the gen_statem process is started with start_link/3, no ServerName is provided and the gen_statem process is not registered.

Module is the name of the callback module.

Args is an arbitrary term that is passed as the argument to Module:init/1.

Start options in Opts

{timeout, Time} - The gen_statem process is allowed to spend Time milliseconds before returning from Module:init/1, or it is terminated and this start function returns {error, timeout}.

{spawn_opt, SpawnOpts} - SpawnOpts is passed as option list to erlang:spawn_opt/2, which is used to spawn the gen_statem process. See proc_lib:start_spawn_option/0.

Note

Using spawn option monitor is not allowed, it causes a badarg failure.

{hibernate_after, HibernateAfterTimeout} - When the gen_statem process waits for a message, if no message is received within HibernateAfterTimeout milliseconds, the process goes into hibernation automatically (by calling proc_lib:hibernate/3). This option is also allowed for the enter_loop functions.

Note that there is also a transition_option/0 to explicitly hibernate the server from a state callback.

{debug, Dbgs} - Activates debugging through sys. For every entry in Dbgs, the corresponding function in sys is called. This option is also allowed for the enter_loop functions.

Return values

{ok, Pid} - The gen_statem server process was successfully created and initialized. Pid is the pid/0 of the process.

ignore - Module:init/1 returned ignore. The gen_statem process has exited with reason normal.

{error, {already_started, OtherPid}} - A process with the specified ServerName already exists. OtherPid is the pid/0 of that process. The gen_statem process exited with reason normal before calling Module:init/1.

{error, timeout} - Module:init/1 did not return within the start time-out. The gen_statem process has been killed with exit(_, kill).

{error, Reason}

Either Module:init/1 returned {stop, Reason} or failed with reason Reason, The gen_statem process exited with reason Reason.

Or Module:init/1 returned {error, Reason}. The gen_statem process did a graceful exit with reason normal.

If the return value is ignore or {error, _}, the started gen_statem process has terminated. If an 'EXIT' message was delivered to the calling process (due to the process link), that message has been consumed.

Warning

Before OTP 26.0, if the started gen_statem process returned e.g. {stop, Reason} from Module:init/1, this function could return {error, Reason} before the started gen_statem process had terminated, so starting again might fail because VM resources such as the registered name was not yet unregistered, and an 'EXIT' message could arrive later to the process calling this function.

But if the started gen_statem process instead failed during Module:init/1, a process link {'EXIT', Pid, Reason} message caused this function to return {error, Reason}, so the 'EXIT' message had been consumed and the started gen_statem process had terminated.

Since it was impossible to tell the difference between these two cases from start_link/3,4's return value, this inconsistency was cleaned up in OTP 26.0.

start_monitor(Module, Args, Opts)

(since OTP 23.0)

-spec start_monitor(Module :: module(), Args :: term(), Opts :: [start_opt()]) -> start_mon_ret().

Start a server, monitored but neither linked nor registered.

Equivalent to start_monitor/4 except that the gen_statem process is not registered with any name service.

start_monitor(ServerName, Module, Args, Opts)

(since OTP 23.0)

-spec start_monitor(ServerName :: server_name(),
                    Module :: module(),
                    Args :: term(),
                    Opts :: [start_opt()]) ->
                       start_mon_ret().

Start a server, monitored and registered, but not linked.

Creates a standalone gen_statem process according to OTP design principles (using proc_lib primitives), and atomically sets up a monitor to the newly created process.

As the started process does not get linked to the calling process, this start function cannot be used by a supervisor to start a child.

For a description of arguments and return values, see start_link/4, but note that for a succesful start the return value differs since this function returns {ok, {Pid, Mon}}, where Pid is the process identifier of the process, and Mon is the monitor reference for the process. If the start is not successful, the caller will be blocked until the DOWN message has been received and removed from the caller's message queue.

stop(ServerRef)

(since OTP 19.0)

-spec stop(ServerRef :: server_ref()) -> ok.

Equivalent to stop(ServerRef, normal, infinity).

stop(ServerRef, Reason, Timeout)

(since OTP 19.0)

-spec stop(ServerRef :: server_ref(), Reason :: term(), Timeout :: timeout()) -> ok.

Stop a server.

Orders the gen_statem ServerRef to exit with the specified Reason and waits for it to terminate. The gen_statem calls Module:terminate/3 before exiting.

This function returns ok if the server terminates with the expected reason. Any other reason than normal, shutdown, or {shutdown, Term} causes an error report to be issued through logger. An exit signal with the same reason is sent to linked processes and ports. The default Reason is normal.

Timeout is an integer > 0, which specifies how many milliseconds to wait for the server to terminate, or the atom infinity to wait indefinitely. Defaults to infinity. If the server does not terminate within the specified time, the call exits the calling process with reason timeout.

If the process does not exist, the call exits the calling process with reason noproc, or with reason {nodedown, Node} if the connection fails to the remote Node where the server runs.

wait_response(ReqId)

(since OTP 23.0)

-spec wait_response(ReqId) -> Result
                       when
                           ReqId :: request_id(),
                           Response ::
                               {reply, Reply :: term()} | {error, {Reason :: term(), server_ref()}},
                           Result :: Response | timeout.

Equivalent to wait_response(ReqId, infinity).

wait_response(ReqId, WaitTime)

(since OTP 23.0)

-spec wait_response(ReqId, WaitTime) -> Result
                       when
                           ReqId :: request_id(),
                           WaitTime :: response_timeout(),
                           Response ::
                               {reply, Reply :: term()} | {error, {Reason :: term(), server_ref()}},
                           Result :: Response | timeout.

Wait for a request response.

Waits for the response to the request identifier ReqId. The request must have been made by send_request/2 to the gen_statem process. This function must be called from the same process from which send_request/2 was called.

WaitTime specifies how long to wait for a reply. If no reply is received within the specified time, the function returns timeout and no cleanup is done, Thus the function can be invoked repeatedly until a reply is returned.

See call/3 about how the request is handled and the Reply is sent by the gen_statem server.

If the gen_statem server process is dead or dies while this function waits for the reply, it returns an error return with the exit Reason.

The difference between receive_response/2 and wait_response/2 is that receive_response/2 abandons the request at time-out so that a potential future response is ignored, while wait_response/2 does not.

wait_response(ReqIdCollection, WaitTime, Delete)

(since OTP 25.0)

-spec wait_response(ReqIdCollection, WaitTime, Delete) -> Result
                       when
                           ReqIdCollection :: request_id_collection(),
                           WaitTime :: response_timeout(),
                           Delete :: boolean(),
                           Response ::
                               {reply, Reply :: term()} | {error, {Reason :: term(), server_ref()}},
                           Result ::
                               {Response,
                                Label :: term(),
                                NewReqIdCollection :: request_id_collection()} |
                               no_request | timeout.

Wait for any request response in a collection.

Waits for a response in ReqIdCollection. All request identifiers of ReqIdCollection must correspond to requests that have been made using send_request/2 or send_request/4, and all requests must have been made by the process calling this function.

The Label in the response is the Label associated with the request identifier that the response corresponds to. The Label of a request identifier is associated when adding the request id to a collection, or when sending the request using send_request/4.

Compared to wait_response/2, the returned result or exception associated with a specific request identifier will be wrapped in a 3-tuple {Response, Label, NewReqIdCollection}. Response is the value that would have been produced by wait_response/2, Label is the value associated with the specific request identifier and NewReqIdCollection is a possibly modified request identifier collection.

If ReqIdCollection is empty, no_request is returned.

If no response is received before WaitTime has expired, timeout is returned. It is valid to continue waiting for a response as many times as needed up until a response has been received and completed by check_response(), receive_response(), or wait_response().

The difference between receive_response/3 and wait_response/3 is that receive_response/3 abandons requests at time-out so that potential future responses are ignored, while wait_response/3 does not.

If Delete is true, the association with Label has been deleted from ReqIdCollection in the resulting NewReqIdCollection. If Delete is false, NewReqIdCollection will equalReqIdCollection. Note that deleting an association is not for free and that a collection containing already handled requests can still be used by subsequent calls to wait_response/3, check_response/3, and receive_response/3.

However, without deleting handled associations, the above calls will not be able to detect when there are no more outstanding requests to handle, so you will have to keep track of this some other way than relying on a no_request return. Note that if you pass a collection only containing associations of already handled or abandoned requests to this function, it will always block until WaitTime expires and then return timeout.

log_mf_h

An event handler that logs events to disk.

This module is a gen_event handler module that can be installed in any gen_event process. It logs onto disk all events that are sent to an event manager. Each event is written as a binary, which makes the logging very fast. However, a tool such as the Report Browser (rb) must be used to read the files. The events are written to multiple files. When all files have been used, the first one is reused and overwritten. The directory location, the number of files, and the size of each file are configurable. The directory will include one file called index, and report files 1, 2, ....

See Also

gen_event, rb

Summary

Types

args()

Term to be sent to gen_event:add_handler/3.

b()

f()

pred()

Functions

init(Dir, MaxBytes, MaxFiles)

Equivalent to init(Dir, MaxBytes, MaxFiles, fun(_) -> true end).

init(Dir, MaxBytes, MaxFiles, Pred)

Initiates the event handler. Returns Args, which is to be used in a call to gen_event:add_handler(EventMgr, log_mf_h, Args).

Types

args()

-opaque args()

Term to be sent to gen_event:add_handler/3.

b()

(not exported)

-type b() :: non_neg_integer().

f()

(not exported)

-type f() :: 1..255.

pred()

(not exported)

-type pred() :: fun((term()) -> boolean()).

Functions

init(Dir, MaxBytes, MaxFiles)

-spec init(Dir, MaxBytes, MaxFiles) -> Args
              when
                  Dir :: file:filename(),
                  MaxBytes :: non_neg_integer(),
                  MaxFiles :: 1..255,
                  Args :: args().

Equivalent to init(Dir, MaxBytes, MaxFiles, fun(_) -> true end).

init(Dir, MaxBytes, MaxFiles, Pred)

-spec init(Dir, MaxBytes, MaxFiles, Pred) -> Args
              when
                  Dir :: file:filename(),
                  MaxBytes :: non_neg_integer(),
                  MaxFiles :: 1..255,
                  Pred :: fun((Event :: term()) -> boolean()),
                  Args :: args().

Initiates the event handler. Returns Args, which is to be used in a call to gen_event:add_handler(EventMgr, log_mf_h, Args).

Dir specifies which directory to use for the log files. MaxBytes specifies the size of each individual file. MaxFiles specifies how many files are used. Pred is a predicate function used to filter the events. If no predicate function is specified, all events are logged.

pool

Load distribution facility.

This module can be used to run a set of Erlang nodes as a pool of computational processors. It is organized as a master and a set of slave nodes and includes the following features:

The slave nodes send regular reports to the master about their current load.

Queries can be sent to the master to determine which node will have the least load.

The BIF statistics(run_queue) is used for estimating future loads. It returns the length of the queue of ready to run processes in the Erlang runtime system.

The slave nodes are started with the slave module. This effects terminal I/O, file I/O, and code loading.

If the master node fails, the entire pool exits.

Files

.hosts.erlang is used to pick hosts where nodes can be started. For information about format and location of this file, see net_adm:host_file/0.

$HOME/.erlang.slave.out.HOST is used for all extra I/O that can come from the slave nodes on standard I/O. If the startup procedure does not work, this file can indicate the reason.

Summary

Functions

attach(Node)

Ensures that a pool master is running and includes Node in the pool master's pool of nodes.

get_node()

Returns the node with the expected lowest future load.

get_nodes()

Returns a list of the current member nodes of the pool.

pspawn(Mod, Fun, Args)

Spawns a process on the pool node that is expected to have the lowest future load.

pspawn_link(Mod, Fun, Args)

Spawns and links to a process on the pool node that is expected to have the lowest future load.

start(Name)

Equivalent to start(Name, []).

start(Name, Args)

Starts a new pool.

stop()

Stops the pool and kills all the slave nodes.

Functions

attach(Node)

-spec attach(Node) -> already_attached | attached when Node :: node().

Ensures that a pool master is running and includes Node in the pool master's pool of nodes.

get_node()

-spec get_node() -> node().

Returns the node with the expected lowest future load.

get_nodes()

-spec get_nodes() -> [node()].

Returns a list of the current member nodes of the pool.

pspawn(Mod, Fun, Args)

-spec pspawn(Mod, Fun, Args) -> pid() when Mod :: module(), Fun :: atom(), Args :: [term()].

Spawns a process on the pool node that is expected to have the lowest future load.

pspawn_link(Mod, Fun, Args)

-spec pspawn_link(Mod, Fun, Args) -> pid() when Mod :: module(), Fun :: atom(), Args :: [term()].

Spawns and links to a process on the pool node that is expected to have the lowest future load.

start(Name)

-spec start(Name) -> Nodes when Name :: atom(), Nodes :: [node()].

Equivalent to start(Name, []).

start(Name, Args)

-spec start(Name, Args) -> Nodes when Name :: atom(), Args :: string(), Nodes :: [node()].

Starts a new pool.

The file .hosts.erlang is read to find host names where the pool nodes can be started; see section Files. The startup procedure fails if the file is not found.

The slave nodes are started with slave:start/2,3, passing along Name and, if provided, Args. Name is used as the first part of the node names, Args is used to specify command-line arguments.

Access rights must be set so that all nodes in the pool have the authority to access each other.

The function is synchronous and all the nodes, and all the system servers, are running when it returns a value.

stop()

-spec stop() -> stopped.

Stops the pool and kills all the slave nodes.

proc_lib

Functions for asynchronous and synchronous start of processes adhering to the OTP design principles.

This module is used to start processes adhering to the OTP Design Principles. Specifically, the functions in this module are used by the OTP standard behaviors (for example, gen_server and gen_statem) when starting new processes. The functions can also be used to start special processes, user-defined processes that comply to the OTP design principles. For an example, see section sys and proc_lib in OTP Design Principles.

Some useful information is initialized when a process starts. The registered names, or the process identifiers, of the parent process, and the parent ancestors, are stored together with information about the function initially called in the process.

While in "plain Erlang", a process is said to terminate normally only for exit reason normal, a process started using proc_lib is also said to terminate normally if it exits with reason shutdown or {shutdown,Term}. shutdown is the reason used when an application (supervision tree) is stopped.

When a process that is started using proc_lib terminates abnormally (that is, with another exit reason than normal, shutdown, or {shutdown,Term}), a crash report is generated, which is written to terminal by the default logger handler setup by Kernel. For more information about how crash reports were logged prior to Erlang/OTP 21.0, see SASL Error Logging in the SASL User's Guide.

Unlike in "plain Erlang", proc_lib processes will not generate error reports, which are written to the terminal by the emulator. All exceptions are converted to exits which are ignored by the default logger handler.

The crash report contains the previously stored information, such as ancestors and initial function, the termination reason, and information about other processes that terminate as a result of this process terminating.

See Also

logger

Summary

Types

dict_or_pid()

exception()

An exception passed to init_fail/3. See erlang:raise/3 for a description of Class, Reason and Stacktrace.

spawn_option()

Equivalent to erlang:spawn_opt_option/0.

start_spawn_option()

A restricted set of spawn options. Most notably monitor is not part of these options.

Functions

format(CrashReport)

Equivalent to format(CrashReport, latin1).

format(CrashReport, Encoding)

format(CrashReport, Encoding, Depth)

get_label(Pid)

Returns either undefined or the label for the process Pid set with proc_lib:set_label/1.

hibernate(Module, Function, Args)

This function does the same as (and does call) the hibernate/3 BIF, but ensures that exception handling and logging continues to work as expected when the process wakes up.

init_ack(Ret)

Equivalent to init_ack(Parent, Ret) where Parent is the process that called start/5.

init_ack(Parent, Ret)

This function must only be used by a process that has been started by a start[_link|_monitor]/3,4,5 function. It tells Parent that the process has initialized itself and started.

init_fail(Return, Exception)

Equivalent to init_fail(Parent, Return, Exception) where Parent is the process that called start/5.

init_fail(Parent, Return, Exception)

This function must only be used by a process that has been started by a start[_link|_monitor]/3,4,5 function. It tells Parent that the process has failed to initialize, and immediately raises an exception according to Exception. The start function then returns Ret.

initial_call(Process)

Extracts the initial call of a process that was started using one of the spawn or start functions in this module. Process can either be a pid, an integer tuple (from which a pid can be created), or the process information of a process Pid fetched through an erlang:process_info(Pid) function call.

set_label(Label)

Set a label for the current process. The primary purpose is to aid in debugging unregistered processes. The process label can be used in tools and crash reports to identify processes but it doesn't have to be unique or an atom, as a registered name needs to be. The process label can be any term, for example {worker_process, 1..N}.

spawn(Fun)

Equivalent to spawn(erlang, apply, [Fun, []]).

spawn(Node, Fun)

Equivalent to spawn(Node, erlang, apply, [Fun, []]).

spawn(Module, Function, Args)

Equivalent to spawn(node(), Module, Function, Args).

spawn(Node, Module, Function, Args)

Spawns a new process and initializes it as described in the beginning of this manual page. The process is spawned using the spawn BIFs.

spawn_link(Fun)

Equivalent to spawn_link(erlang, apply, [Fun, []]).

spawn_link(Node, Fun)

Equivalent to spawn_link(Node, erlang, apply, [Fun, []]).

spawn_link(Module, Function, Args)

Equivalent to spawn_link(node(), Module, Function, Args).

spawn_link(Node, Module, Function, Args)

Spawns a new process and initializes it as described in the beginning of this manual page. The process is spawned using the spawn_link BIFs.

spawn_opt(Fun, SpawnOpts)

Equivalent to spawn_opt(erlang, apply, [Fun, []], SpawnOpts).

spawn_opt(Node, Fun, SpawnOpts)

Equivalent to spawn_opt(Node, erlang, apply, [Fun, []], SpawnOpts).

spawn_opt(Module, Function, Args, SpawnOpts)

Equivalent to spawn_opt(node(), Module, Function, Args, SpawnOpts).

spawn_opt(Node, Module, Function, Args, SpawnOpts)

Spawns a new process and initializes it as described in the beginning of this manual page. The process is spawned using the erlang:spawn_opt BIFs.

start(Module, Function, Args)

Equivalent to start(Module, Function, Args, infinity).

start(Module, Function, Args, Time)

Equivalent to start(Module, Function, Args, Time, []).

start(Module, Function, Args, Time, SpawnOpts)

Starts a new process synchronously. Spawns the process and waits for it to start.

start_link(Module, Function, Args)

Equivalent to start_link(Module, Function, Args, infinity).

start_link(Module, Function, Args, Time)

Equivalent to start_link(Module, Function, Args, Time, []).

start_link(Module, Function, Args, Time, SpawnOpts)

Starts a new process synchronously. Spawns the process and waits for it to start. A link is atomically set on the newly spawned process.

start_monitor(Module, Function, Args)

Equivalent to start_monitor(Module, Function, Args, infinity).

start_monitor(Module, Function, Args, Time)

Equivalent to start_monitor(Module, Function, Args, Time, []).

start_monitor(Module, Function, Args, Time, SpawnOpts)

Starts a new process synchronously. Spawns the process and waits for it to start. A monitor is atomically set on the newly spawned process.

stop(Process)

Equivalent to stop(Process, normal, infinity).

stop(Process, Reason, Timeout)

Orders the process to exit with the specified Reason and waits for it to terminate.

translate_initial_call(Process)

This function is used by functions c:i/0 and c:regs/0 to present process information.

Types

dict_or_pid()

(not exported)

-type dict_or_pid() :: pid() | (ProcInfo :: [_]) | {X :: integer(), Y :: integer(), Z :: integer()}.

exception()

(not exported)

-type exception() ::
          {Class :: error | exit | throw, Reason :: term()} |
          {Class :: error | exit | throw, Reason :: term(), Stacktrace :: erlang:raise_stacktrace()}.

An exception passed to init_fail/3. See erlang:raise/3 for a description of Class, Reason and Stacktrace.

spawn_option()

-type spawn_option() :: erlang:spawn_opt_option().

Equivalent to erlang:spawn_opt_option/0.

start_spawn_option()

-type start_spawn_option() ::
          link |
          {priority, erlang:priority_level()} |
          {fullsweep_after, non_neg_integer()} |
          {min_heap_size, non_neg_integer()} |
          {min_bin_vheap_size, non_neg_integer()} |
          {max_heap_size, erlang:max_heap_size()} |
          {message_queue_data, erlang:message_queue_data()}.

A restricted set of spawn options. Most notably monitor is not part of these options.

Functions

format(CrashReport)

-spec format(CrashReport) -> string() when CrashReport :: [term()].

Equivalent to format(CrashReport, latin1).

format(CrashReport, Encoding)

(since OTP R16B)

-spec format(CrashReport, Encoding) -> string()
                when CrashReport :: [term()], Encoding :: latin1 | unicode | utf8.

Note

This function is deprecated in the sense that the error_logger is no longer the preferred interface for logging in Erlang/OTP. A new logging API was added in Erlang/OTP 21.0, but legacy error_logger handlers can still be used. New Logger handlers do not need to use this function, since the formatting callback (report_cb) is included as metadata in the log event.

This function can be used by a user-defined legacy error_logger event handler to format a crash report. The crash report is sent using logger, and the event to be handled is of the format {error_report, GL, {Pid, crash_report, CrashReport}}, where GL is the group leader pid of process Pid that sent the crash report.

format(CrashReport, Encoding, Depth)

(since OTP 18.1)

-spec format(CrashReport, Encoding, Depth) -> string()
                when
                    CrashReport :: [term()],
                    Encoding :: latin1 | unicode | utf8,
                    Depth :: unlimited | pos_integer().

Note

This function is deprecated in the sense that the error_logger is no longer the preferred interface for logging in Erlang/OTP. A new logging API was added in Erlang/OTP 21.0, but legacy error_logger handlers can still be used. New Logger handlers do not need to used this function, since the formatting callback (report_cb) is included as metadata in the log event.

This function can be used by a user-defined legacy error_logger event handler to format a crash report. When Depth is specified as a positive integer, it is used in the format string to limit the output as follows: io_lib:format("~P", [Term,Depth]).

get_label(Pid)

(since OTP 27.0)

-spec get_label(Pid) -> undefined | term() when Pid :: pid().

Returns either undefined or the label for the process Pid set with proc_lib:set_label/1.

hibernate(Module, Function, Args)

-spec hibernate(Module, Function, Args) -> no_return()
                   when Module :: module(), Function :: atom(), Args :: [term()].

This function does the same as (and does call) the hibernate/3 BIF, but ensures that exception handling and logging continues to work as expected when the process wakes up.

Always use this function instead of the BIF for processes started using proc_lib functions.

init_ack(Ret)

-spec init_ack(Ret) -> ok when Ret :: term().

Equivalent to init_ack(Parent, Ret) where Parent is the process that called start/5.

init_ack(Parent, Ret)

-spec init_ack(Parent, Ret) -> ok when Parent :: pid(), Ret :: term().

This function must only be used by a process that has been started by a start[_link|_monitor]/3,4,5 function. It tells Parent that the process has initialized itself and started.

Function init_ack/1 uses the parent value previously stored by the start function used.

If neither this function nor init_fail/2,3 is called by the started process, the start function returns an error tuple when the started process exits, or when the start function time-out (if used) has passed, see start/3,4,5.

Warning

Do not use this function to return an error indicating that the process start failed. When doing so the start function can return before the failing process has exited, which may block VM resources required for a new start attempt to succeed. Use init_fail/2,3 for that purpose.

The following example illustrates how this function and proc_lib:start_link/3 are used:

-module(my_proc).
-export([start_link/0]).
-export([init/1]).

start_link() ->
    proc_lib:start_link(my_proc, init, [self()]).

init(Parent) ->
    case do_initialization() of
        ok ->
            proc_lib:init_ack(Parent, {ok, self()});
        {error, Reason} ->
            exit(Reason)
    end,
    loop().

...

init_fail(Return, Exception)

(since OTP 26.0)

-spec init_fail(Return :: term(), Exception :: exception()) -> no_return().

Equivalent to init_fail(Parent, Return, Exception) where Parent is the process that called start/5.

init_fail(Parent, Return, Exception)

(since OTP 26.0)

-spec init_fail(Parent :: pid(), Return :: term(), Exception :: exception()) -> no_return().

This function must only be used by a process that has been started by a start[_link|_monitor]/3,4,5 function. It tells Parent that the process has failed to initialize, and immediately raises an exception according to Exception. The start function then returns Ret.

See erlang:raise/3 for a description of Class, Reason and Stacktrace.

Warning

Do not consider catching the exception from this function. That would defeat its purpose. A process started by a start[_link|_monitor]/3,4,5 function should end in a value (that will be ignored) or an exception that will be handled by this module. See Description.

If neither this function nor init_ack/1,2 is called by the started process, the start function returns an error tuple when the started process exits, or when the start function time-out (if used) has passed, see start/3,4,5.

The following example illustrates how this function and proc_lib:start_link/3 can be used:

-module(my_proc).
-export([start_link/0]).
-export([init/1]).

start_link() ->
    proc_lib:start_link(my_proc, init, [self()]).

init(Parent) ->
    case do_initialization() of
        ok ->
            proc_lib:init_ack(Parent, {ok, self()});
        {error, Reason} = Error ->
            proc_lib:init_fail(Parent, Error, {exit, normal})
    end,
    loop().

...

initial_call(Process)

-spec initial_call(Process) -> {Module, Function, Args} | false
                      when
                          Process :: dict_or_pid(),
                          Module :: module(),
                          Function :: atom(),
                          Args :: [atom()].

Extracts the initial call of a process that was started using one of the spawn or start functions in this module. Process can either be a pid, an integer tuple (from which a pid can be created), or the process information of a process Pid fetched through an erlang:process_info(Pid) function call.

Note

The list Args no longer contains the arguments, but the same number of atoms as the number of arguments; the first atom is 'Argument__1', the second 'Argument__2', and so on. The reason is that the argument list could waste a significant amount of memory, and if the argument list contained funs, it could be impossible to upgrade the code for the module.

If the process was spawned using a fun, initial_call/1 no longer returns the fun, but the module, function for the local function implementing the fun, and the arity, for example, {some_module,-work/3-fun-0-,0} (meaning that the fun was created in function some_module:work/3). The reason is that keeping the fun would prevent code upgrade for the module, and that a significant amount of memory could be wasted.

set_label(Label)

(since OTP 27.0)

-spec set_label(Label) -> ok when Label :: term().

Set a label for the current process. The primary purpose is to aid in debugging unregistered processes. The process label can be used in tools and crash reports to identify processes but it doesn't have to be unique or an atom, as a registered name needs to be. The process label can be any term, for example {worker_process, 1..N}.

Use proc_lib:get_label/1 to lookup the process description.

spawn(Fun)

-spec spawn(Fun) -> pid() when Fun :: function().

Equivalent to spawn(erlang, apply, [Fun, []]).

spawn(Node, Fun)

-spec spawn(Node, Fun) -> pid() when Node :: node(), Fun :: function().

Equivalent to spawn(Node, erlang, apply, [Fun, []]).

spawn(Module, Function, Args)

-spec spawn(Module, Function, Args) -> pid()
               when Module :: module(), Function :: atom(), Args :: [term()].

Equivalent to spawn(node(), Module, Function, Args).

spawn(Node, Module, Function, Args)

-spec spawn(Node, Module, Function, Args) -> pid()
               when Node :: node(), Module :: module(), Function :: atom(), Args :: [term()].

Spawns a new process and initializes it as described in the beginning of this manual page. The process is spawned using the spawn BIFs.

spawn_link(Fun)

-spec spawn_link(Fun) -> pid() when Fun :: function().

Equivalent to spawn_link(erlang, apply, [Fun, []]).

spawn_link(Node, Fun)

-spec spawn_link(Node, Fun) -> pid() when Node :: node(), Fun :: function().

Equivalent to spawn_link(Node, erlang, apply, [Fun, []]).

spawn_link(Module, Function, Args)

-spec spawn_link(Module, Function, Args) -> pid()
                    when Module :: module(), Function :: atom(), Args :: [term()].

Equivalent to spawn_link(node(), Module, Function, Args).

spawn_link(Node, Module, Function, Args)

-spec spawn_link(Node, Module, Function, Args) -> pid()
                    when Node :: node(), Module :: module(), Function :: atom(), Args :: [term()].

Spawns a new process and initializes it as described in the beginning of this manual page. The process is spawned using the spawn_link BIFs.

spawn_opt(Fun, SpawnOpts)

-spec spawn_opt(Fun, SpawnOpts) -> pid() | {pid(), reference()}
                   when Fun :: function(), SpawnOpts :: [erlang:spawn_opt_option()].

Equivalent to spawn_opt(erlang, apply, [Fun, []], SpawnOpts).

spawn_opt(Node, Fun, SpawnOpts)

-spec spawn_opt(Node, Fun, SpawnOpts) -> pid() | {pid(), reference()}
                   when Node :: node(), Fun :: function(), SpawnOpts :: [erlang:spawn_opt_option()].

Equivalent to spawn_opt(Node, erlang, apply, [Fun, []], SpawnOpts).

spawn_opt(Module, Function, Args, SpawnOpts)

-spec spawn_opt(Module, Function, Args, SpawnOpts) -> pid() | {pid(), reference()}
                   when
                       Module :: module(),
                       Function :: atom(),
                       Args :: [term()],
                       SpawnOpts :: [erlang:spawn_opt_option()].

Equivalent to spawn_opt(node(), Module, Function, Args, SpawnOpts).

spawn_opt(Node, Module, Function, Args, SpawnOpts)

-spec spawn_opt(Node, Module, Function, Args, SpawnOpts) -> pid() | {pid(), reference()}
                   when
                       Node :: node(),
                       Module :: module(),
                       Function :: atom(),
                       Args :: [term()],
                       SpawnOpts :: [erlang:spawn_opt_option()].

Spawns a new process and initializes it as described in the beginning of this manual page. The process is spawned using the erlang:spawn_opt BIFs.

start(Module, Function, Args)

-spec start(Module, Function, Args) -> Ret
               when
                   Module :: module(),
                   Function :: atom(),
                   Args :: [term()],
                   Ret :: term() | {error, Reason :: term()}.

Equivalent to start(Module, Function, Args, infinity).

start(Module, Function, Args, Time)

-spec start(Module, Function, Args, Time) -> Ret
               when
                   Module :: module(),
                   Function :: atom(),
                   Args :: [term()],
                   Time :: timeout(),
                   Ret :: term() | {error, Reason :: term()}.

Equivalent to start(Module, Function, Args, Time, []).

start(Module, Function, Args, Time, SpawnOpts)

-spec start(Module, Function, Args, Time, SpawnOpts) -> Ret
               when
                   Module :: module(),
                   Function :: atom(),
                   Args :: [term()],
                   Time :: timeout(),
                   SpawnOpts :: [start_spawn_option()],
                   Ret :: term() | {error, Reason :: term()}.

Starts a new process synchronously. Spawns the process and waits for it to start.

To indicate a succesful start, the started process must call init_ack(Parent, Ret) where Parent is the process that evaluates this function, or init_ack(Ret). Ret is then returned by this function.

If the process fails to start, it must fail; preferably by calling init_fail(Parent, Ret, Exception) where Parent is the process that evaluates this function, or init_fail(Ret, Exception). Ret is then returned by this function, and the started process fails with Exception.

If the process instead fails before calling init_ack/1,2 or init_fail/2,3, this function returns {error, Reason} where Reason depends a bit on the exception just like for a process link {'EXIT',Pid,Reason} message.

If Time is specified as an integer, this function waits for Time milliseconds for the new process to call init_ack/1,2 or init_fail/2,3, otherwise the process gets killed and Ret = {error, timeout} is returned.

Argument SpawnOpts, if specified, is passed as the last argument to the spawn_opt/4 BIF.

Note

Using spawn option monitor is not allowed. It causes the function to fail with reason badarg.

Using spawn option link will set a link to the spawned process, just like start_link/3,4,5.

start_link(Module, Function, Args)

-spec start_link(Module, Function, Args) -> Ret
                    when
                        Module :: module(),
                        Function :: atom(),
                        Args :: [term()],
                        Ret :: term() | {error, Reason :: term()}.

Equivalent to start_link(Module, Function, Args, infinity).

start_link(Module, Function, Args, Time)

-spec start_link(Module, Function, Args, Time) -> Ret
                    when
                        Module :: module(),
                        Function :: atom(),
                        Args :: [term()],
                        Time :: timeout(),
                        Ret :: term() | {error, Reason :: term()}.

Equivalent to start_link(Module, Function, Args, Time, []).

start_link(Module, Function, Args, Time, SpawnOpts)

-spec start_link(Module, Function, Args, Time, SpawnOpts) -> Ret
                    when
                        Module :: module(),
                        Function :: atom(),
                        Args :: [term()],
                        Time :: timeout(),
                        SpawnOpts :: [start_spawn_option()],
                        Ret :: term() | {error, Reason :: term()}.

Starts a new process synchronously. Spawns the process and waits for it to start. A link is atomically set on the newly spawned process.

Note

If the started process gets killed or crashes with a reason that is not normal, the process link will kill the calling process so this function does not return, unless the calling process traps exits. For example, if this function times out it will kill the spawned process, and then the link might kill the calling process.

Besides setting a link on the spawned process this function behaves like start/5.

When the calling process traps exits; if this function returns due to the spawned process exiting (any error return), this function receives (consumes) the 'EXIT' message, also when this function times out and kills the spawned process.

Note

Using spawn option monitor is not allowed. It causes the function to fail with reason badarg.

start_monitor(Module, Function, Args)

(since OTP 23.0)

-spec start_monitor(Module, Function, Args) -> {Ret, Mon}
                       when
                           Module :: module(),
                           Function :: atom(),
                           Args :: [term()],
                           Mon :: reference(),
                           Ret :: term() | {error, Reason :: term()}.

Equivalent to start_monitor(Module, Function, Args, infinity).

start_monitor(Module, Function, Args, Time)

(since OTP 23.0)

-spec start_monitor(Module, Function, Args, Time) -> {Ret, Mon}
                       when
                           Module :: module(),
                           Function :: atom(),
                           Args :: [term()],
                           Time :: timeout(),
                           Mon :: reference(),
                           Ret :: term() | {error, Reason :: term()}.

Equivalent to start_monitor(Module, Function, Args, Time, []).

start_monitor(Module, Function, Args, Time, SpawnOpts)

(since OTP 23.0)

-spec start_monitor(Module, Function, Args, Time, SpawnOpts) -> {Ret, Mon}
                       when
                           Module :: module(),
                           Function :: atom(),
                           Args :: [term()],
                           Time :: timeout(),
                           SpawnOpts :: [start_spawn_option()],
                           Mon :: reference(),
                           Ret :: term() | {error, Reason :: term()}.

Starts a new process synchronously. Spawns the process and waits for it to start. A monitor is atomically set on the newly spawned process.

Besides setting a monitor on the spawned process this function behaves like start/5.

The return value is {Ret, Mon} where Ret corresponds to the Ret argument in the call to init_ack/1,2 or init_fail/2,3, and Mon is the monitor reference of the monitor that has been set up.

If this function returns due to the spawned process exiting, that is returns any error value, a 'DOWN' message will be delivered to the calling process, also when this function times out and kills the spawned process.

Note

Using spawn option monitor is not allowed. It causes the function to fail with reason badarg.

Using spawn option link will set a link to the spawned process, just like start_link/3,4,5.

stop(Process)

(since OTP 18.0)

-spec stop(Process) -> ok when Process :: pid() | RegName | {RegName, node()}, RegName :: atom().

Equivalent to stop(Process, normal, infinity).

stop(Process, Reason, Timeout)

(since OTP 18.0)

-spec stop(Process, Reason, Timeout) -> ok
              when
                  Process :: pid() | RegName | {RegName, node()},
                  RegName :: atom(),
                  Reason :: term(),
                  Timeout :: timeout().

Orders the process to exit with the specified Reason and waits for it to terminate.

Returns ok if the process exits with the specified Reason within Timeout milliseconds.

If the call times out, a timeout exception is raised.

If the process does not exist, a noproc exception is raised.

The implementation of this function is based on the terminate system message, and requires that the process handles system messages correctly. For information about system messages, see sys and section sys and proc_lib in OTP Design Principles.

translate_initial_call(Process)

-spec translate_initial_call(Process) -> {Module, Function, Arity}
                                when
                                    Process :: dict_or_pid(),
                                    Module :: module(),
                                    Function :: atom(),
                                    Arity :: byte().

This function is used by functions c:i/0 and c:regs/0 to present process information.

This function extracts the initial call of a process that was started using one of the spawn or start functions in this module, and translates it to more useful information. Process can either be a pid, an integer tuple (from which a pid can be created), or the process information of a process Pid fetched through an erlang:process_info(Pid) function call.

If the initial call is to one of the system-defined behaviors such as gen_server or gen_event, it is translated to more useful information. If a gen_server is spawned, the returned Module is the name of the callback module and Function is init (the function that initiates the new server).

A supervisor and a supervisor_bridge are also gen_server processes. To return information that this process is a supervisor and the name of the callback module, Module is supervisor and Function is the name of the supervisor callback module. Arity is 1, as the init/1 function is called initially in the callback module.

By default, {proc_lib,init_p,5} is returned if no information about the initial call can be found. It is assumed that the caller knows that the process has been spawned with the proc_lib module.

supervisor behaviour

Generic supervisor behavior.

This behavior module provides a supervisor, a process that supervises other processes called child processes. A child process can either be another supervisor or a worker process. Worker processes are normally implemented using one of the gen_event, gen_server, or gen_statem behaviors. A supervisor implemented using this module has a standard set of interface functions and includes functionality for tracing and error reporting. Supervisors are used to build a hierarchical process structure called a supervision tree, a nice way to structure a fault-tolerant application. For more information, see Supervisor Behaviour in OTP Design Principles.

A supervisor expects the definition of which child processes to supervise to be specified in a callback module exporting a predefined set of functions.

Unless otherwise stated, all functions in this module fail if the specified supervisor does not exist or if bad arguments are specified.

Supervision Principles

The supervisor is responsible for starting, stopping, and monitoring its child processes. The basic idea of a supervisor is that it must keep its child processes alive by restarting them when necessary.

The children of a supervisor are defined as a list of child specifications. When the supervisor is started, the child processes are started in order from left to right according to this list. When the supervisor is going to terminate, it first terminates its child processes in reversed start order, from right to left.

Supervisor flags

The supervisor properties are defined by the supervisor flags. The type definition for the supervisor flags is as follows:

sup_flags() = #{strategy => strategy(),           % optional
                intensity => non_neg_integer(),   % optional
                period => pos_integer(),          % optional
                hibernate_after => timeout(),     % optional, available since OTP 28.0
                auto_shutdown => auto_shutdown()} % optional

Restart Strategies

A supervisor can have one of the following restart strategies specified with the strategy key in the above map:

one_for_one - If one child process terminates and is to be restarted, only that child process is affected. This is the default restart strategy.

one_for_all - If one child process terminates and is to be restarted, all other child processes are terminated and then all child processes are restarted.

rest_for_one - If one child process terminates and is to be restarted, the 'rest' of the child processes (that is, the child processes after the terminated child process in the start order) are terminated. Then the terminated child process and all child processes after it are restarted.

simple_one_for_one - A simplified one_for_one supervisor, where all child processes are dynamically added instances of the same process type, that is, running the same code.

Functions delete_child/2 and restart_child/2 are invalid for simple_one_for_one supervisors and return {error,simple_one_for_one} if the specified supervisor uses this restart strategy.

Function terminate_child/2 can be used for children under simple_one_for_one supervisors by specifying the child's pid/0 as the second argument. If instead the child specification identifier is used, terminate_child/2 return {error,simple_one_for_one}.

As a simple_one_for_one supervisor can have many children, it shuts them all down asynchronously. This means that the children do their cleanup in parallel, and therefore the order in which they are stopped is not defined.

Restart intensity and period

To prevent a supervisor from getting into an infinite loop of child process terminations and restarts, a maximum restart intensity is defined using two integer values specified with keys intensity and period in the above map. Assuming the values MaxR for intensity and MaxT for period, then, if more than MaxR restarts occur within MaxT seconds, the supervisor terminates all child processes and then itself. The termination reason for the supervisor itself in that case will be shutdown. intensity defaults to 1 and period defaults to 5.

Hibernate after

In order to save memory, a supervisor, like any other process, can go into hibernation. By default, a simple_one_for_one supervisor will never hibernate, as it is expected its children will come and go at potentially high rates. In counterpart, other strategies rather expect children to be stable and therefore will default to hibernating after a certain period of time of inactivity, in order to be responsive to bursts of restarts and save memory in periods of stability. You can finetune this flag by setting hibernate_after, when for example the supervisor will be regularly queried for which_child/1 or similar and hibernation is to be better controlled.

Automatic Shutdown

A supervisor can be configured to automatically shut itself down with exit reason shutdown when significant children terminate with the auto_shutdown key in the above map:

never - Automic shutdown is disabled. This is the default setting.

With auto_shutdown set to never, child specs with the significant flag set to true are considered invalid and will be rejected.

any_significant - The supervisor will shut itself down when any significant child terminates, that is, when a transient significant child terminates normally or when a temporary significant child terminates normally or abnormally.

all_significant - The supervisor will shut itself down when all significant children have terminated, that is, when the last active significant child terminates. The same rules as for any_significant apply.

For more information, see the section Automatic Shutdown in Supervisor Behavior in OTP Design Principles.

Warning

The automatic shutdown feature appeared in OTP 24.0, but applications using this feature will also compile and run with older OTP versions.

However, such applications, when compiled with an OTP version that predates the appearance of the automatic shutdown feature, will leak processes because the automatic shutdowns they rely on will not happen.

It is up to implementors to take proper precautions if they expect that their applications may be compiled with older OTP versions.

Child specification

The type definition of a child specification is as follows:

child_spec() = #{id => child_id(),             % mandatory
                 start => mfargs(),            % mandatory
                 restart => restart(),         % optional
                 significant => significant(), % optional
                 shutdown => shutdown(),       % optional
                 type => worker(),             % optional
                 modules => modules()}         % optional

The old tuple format is kept for backwards compatibility, see child_spec/0, but the map is preferred.

id is used to identify the child specification internally by the supervisor.

The id key is mandatory.

Notice that this identifier on occations has been called "name". As far as possible, the terms "identifier" or "id" are now used but to keep backward compatibility, some occurences of "name" can still be found, for example in error messages.

start defines the function call used to start the child process. It must be a module-function-arguments tuple {M,F,A} used as apply(M,F,A).

The start function must create and link to the child process, and must return {ok,Child} or {ok,Child,Info}, where Child is the pid of the child process and Info any term that is ignored by the supervisor.

The start function can also return ignore if the child process for some reason cannot be started, in which case the child specification is kept by the supervisor (unless it is a temporary child) but the non-existing child process is ignored.

If something goes wrong, the function can also return an error tuple {error,Error}.

Notice that the start_link functions of the different behavior modules fulfill the above requirements.

The start key is mandatory.

restart defines when a terminated child process must be restarted. A permanent child process is always restarted. A temporary child process is never restarted (even when the supervisor's restart strategy is rest_for_one or one_for_all and a sibling's death causes the temporary process to be terminated). A transient child process is restarted only if it terminates abnormally, that is, with another exit reason than normal, shutdown, or {shutdown,Term}.

The restart key is optional. If it is not specified, it defaults to permanent.

significant defines if a child is considered significant for automatic self-shutdown of the supervisor.

Setting this option to true when the restart type is permanent is invalid. Also, it is considered invalid to start children with this option set to true in a supervisor when the auto_shutdown supervisor flag is set to never.

The significant key is optional. If it is not specified, it defaults to false.

shutdown defines how a child process must be terminated. brutal_kill means that the child process is unconditionally terminated using exit(Child,kill). An integer time-out value means that the supervisor tells the child process to terminate by calling exit(Child,shutdown) and then wait for an exit signal with reason shutdown back from the child process. If no exit signal is received within the specified number of milliseconds, the child process is unconditionally terminated using exit(Child,kill).

If the child process is another supervisor, the shutdown time must be set to infinity to give the subtree ample time to shut down.

Warning

Setting the shutdown time to anything other than infinity for a child of type supervisor can cause a race condition where the child in question unlinks its own children, but fails to terminate them before it is killed.

It is also allowed to set it to infinity, if the child process is a worker.

Warning

Be careful when setting the shutdown time to infinity when the child process is a worker. Because, in this situation, the termination of the supervision tree depends on the child process, it must be implemented in a safe way and its cleanup procedure must always return.

Notice that all child processes implemented using the standard OTP behavior modules automatically adhere to the shutdown protocol.

The shutdown key is optional. If it is not specified, it defaults to 5000 if the child is of type worker and it defaults to infinity if the child is of type supervisor.

type specifies if the child process is a supervisor or a worker.

The type key is optional. If it is not specified, it defaults to worker.

modules is used by the release handler during code replacement to determine which processes are using a certain module. As a rule of thumb, if the child process is a supervisor, gen_server or, gen_statem, this is to be a list with one element [Module], where Module is the callback module. If the child process is an event manager (gen_event) with a dynamic set of callback modules, value dynamic must be used. For more information about release handling, see Release Handling in OTP Design Principles.

The modules key is optional. If it is not specified, it defaults to [M], where M comes from the child's start {M,F,A}.

Internally, the supervisor also keeps track of the pid Child of the child process, or undefined if no pid exists.

See Also

gen_event, gen_statem, gen_server, sys

Summary

Types

auto_shutdown()

child()

child_id()

Not a pid/0.

child_rec()

child_spec()

The tuple format is kept for backward compatibility only. A map is preferred; see more details above.

children()

mfargs()

Value undefined for A (the argument list) is only to be used internally in supervisor. If the restart type of the child is temporary, the process is never to be restarted and therefore there is no need to store the real argument list. Value undefined is then stored instead.

modules()

restart()

shutdown()

significant()

startchild_err()

startchild_ret()

startlink_err()

startlink_ret()

strategy()

sup_flags()

The tuple format is kept for backward compatibility only. A map is preferred; see more details above.

sup_name()

Name specification to use when starting a supervisor. See function start_link/2,3 and the type sup_ref/0 below.

sup_ref()

Supervisor specification to use when addressing a supervisor. See count_children/1, delete_child/2, get_childspec/2, restart_child/2, start_child/2, terminate_child/2, which_children/1 and the type sup_name/0 above.

worker()

Callbacks

init(Args)

Whenever a supervisor is started using start_link/2,3, this function is called by the new process to find out about restart strategy, maximum restart intensity, and child specifications.

Functions

check_childspecs(ChildSpecs)

Equivalent to check_childspecs(ChildSpecs, undefined).

check_childspecs(ChildSpecs, AutoShutdown)

Takes a list of child specification as argument and returns ok if all of them are syntactically correct, otherwise {error,Error}.

count_children(SupRef)

Returns a property list containing the counts for each of the following elements of the supervisor's child specifications and managed processes

delete_child(SupRef, Id)

Tells supervisor SupRef to delete the child specification identified by Id. The corresponding child process must not be running. Use terminate_child/2 to terminate it.

get_childspec(SupRef, Id)

Returns the child specification map for the child identified by Id under supervisor SupRef. The returned map contains all keys, both mandatory and optional.

restart_child(SupRef, Id)

Tells supervisor SupRef to restart a child process corresponding to the child specification identified by Id. The child specification must exist, and the corresponding child process must not be running.

start_child(Supervisor, ChildSpecOrExtraArgs)

Dynamically adds a child specification to supervisor SupRef, which starts the corresponding child process.

start_link(Module, Args)

Creates a nameless supervisor process as part of a supervision tree.

start_link(SupName, Module, Args)

Creates a supervisor process as part of a supervision tree.

stop(SupRef)

Equivalent to stop(SupRef, normal, infinity).

stop(SupRef, Reason, Timeout)

Stop a supervisor.

terminate_child(SupRef, Id)

Tells supervisor SupRef to terminate the specified child.

which_child(SupRef, Id)

Returns information about the child specification and child process identified by the given Id.

which_children(SupRef)

Returns a newly created list with information about all child specifications and child processes belonging to supervisor SupRef.

Types

auto_shutdown()

(not exported)

-type auto_shutdown() :: never | any_significant | all_significant.

child()

(not exported)

-type child() :: undefined | pid().

child_id()

(not exported)

-type child_id() :: term().

Not a pid/0.

child_rec()

(not exported)

-type child_rec() ::
          #child{pid :: child() | {restarting, pid() | undefined} | [pid()],
                 id :: child_id(),
                 mfargs :: mfargs(),
                 restart_type :: restart(),
                 significant :: significant(),
                 shutdown :: shutdown(),
                 child_type :: worker(),
                 modules :: modules()}.

child_spec()

-type child_spec() ::
          #{id := child_id(),
            start := mfargs(),
            restart => restart(),
            significant => significant(),
            shutdown => shutdown(),
            type => worker(),
            modules => modules()} |
          {Id :: child_id(),
           StartFunc :: mfargs(),
           Restart :: restart(),
           Shutdown :: shutdown(),
           Type :: worker(),
           Modules :: modules()}.

The tuple format is kept for backward compatibility only. A map is preferred; see more details above.

children()

(not exported)

-type children() :: {Ids :: [child_id()], Db :: #{child_id() => child_rec()}}.

mfargs()

(not exported)

-type mfargs() :: {M :: module(), F :: atom(), A :: [term()] | undefined}.

Value undefined for A (the argument list) is only to be used internally in supervisor. If the restart type of the child is temporary, the process is never to be restarted and therefore there is no need to store the real argument list. Value undefined is then stored instead.

modules()

(not exported)

-type modules() :: [module()] | dynamic.

restart()

(not exported)

-type restart() :: permanent | transient | temporary.

shutdown()

(not exported)

-type shutdown() :: brutal_kill | timeout().

significant()

(not exported)

-type significant() :: boolean().

startchild_err()

-type startchild_err() :: already_present | {already_started, Child :: child()} | term().

startchild_ret()

-type startchild_ret() ::
          {ok, Child :: child()} | {ok, Child :: child(), Info :: term()} | {error, startchild_err()}.

startlink_err()

-type startlink_err() :: {already_started, pid()} | {shutdown, term()} | term().

startlink_ret()

-type startlink_ret() :: {ok, pid()} | ignore | {error, startlink_err()}.

strategy()

-type strategy() :: one_for_all | one_for_one | rest_for_one | simple_one_for_one.

sup_flags()

-type sup_flags() ::
          #{strategy => strategy(),
            intensity => non_neg_integer(),
            period => pos_integer(),
            auto_shutdown => auto_shutdown(),
            hibernate_after => timeout()} |
          {RestartStrategy :: strategy(), Intensity :: non_neg_integer(), Period :: pos_integer()}.

The tuple format is kept for backward compatibility only. A map is preferred; see more details above.

sup_name()

-type sup_name() ::
          {local, Name :: atom()} | {global, Name :: term()} | {via, Module :: module(), Name :: any()}.

Name specification to use when starting a supervisor. See function start_link/2,3 and the type sup_ref/0 below.

{local,LocalName} - Register the supervisor locally as LocalName using register/2.

{global,GlobalName} - Register the supervisor process id globally as GlobalName using global:register_name/2.

{via,RegMod,ViaName} - Register the supervisor process with the registry represented by RegMod. The RegMod callback is to export the functions register_name/2, unregister_name/1, whereis_name/1, and send/2, which are to behave like the corresponding functions in global. Thus, {via,global,GlobalName} is a valid reference equivalent to {global,GlobalName}.

sup_ref()

-type sup_ref() ::
          (Name :: atom()) |
          {Name :: atom(), Node :: node()} |
          {global, Name :: term()} |
          {via, Module :: module(), Name :: any()} |
          pid().

Supervisor specification to use when addressing a supervisor. See count_children/1, delete_child/2, get_childspec/2, restart_child/2, start_child/2, terminate_child/2, which_children/1 and the type sup_name/0 above.

It can be:

pid/0 - The supervisor's process identifier.

LocalName - The supervisor is locally registered as LocalName with register/2.

{Name,Node} - The supervisor is locally registered on another node.

{global,GlobalName} - The supervisor is globally registered in global.

{via,RegMod,ViaName} - The supervisor is registered in an alternative process registry. The registry callback module RegMod is to export functions register_name/2, unregister_name/1, whereis_name/1, and send/2, which are to behave like the corresponding functions in global. Thus, {via,global,GlobalName} is the same as {global,GlobalName}.

worker()

(not exported)

-type worker() :: worker | supervisor.

Callbacks

init(Args)

-callback init(Args :: term()) -> {ok, {SupFlags :: sup_flags(), [ChildSpec :: child_spec()]}} | ignore.

Whenever a supervisor is started using start_link/2,3, this function is called by the new process to find out about restart strategy, maximum restart intensity, and child specifications.

Args is the Args argument provided to the start function.

SupFlags is the supervisor flags defining the restart strategy and maximum restart intensity for the supervisor. [ChildSpec] is a list of valid child specifications defining which child processes the supervisor must start and monitor. See the discussion in section Supervision Principles earlier.

Notice that when the restart strategy is simple_one_for_one, the list of child specifications must be a list with one child specification only. (The child specification identifier is ignored.) No child process is then started during the initialization phase, but all children are assumed to be started dynamically using start_child/2.

The function can also return ignore.

Notice that this function can also be called as a part of a code upgrade procedure. Therefore, the function is not to have any side effects. For more information about code upgrade of supervisors, see section Changing a Supervisor in OTP Design Principles.

Functions

check_childspecs(ChildSpecs)

-spec check_childspecs(ChildSpecs) -> Result
                          when ChildSpecs :: [child_spec()], Result :: ok | {error, Error :: term()}.

Equivalent to check_childspecs(ChildSpecs, undefined).

check_childspecs(ChildSpecs, AutoShutdown)

(since OTP 24.0)

-spec check_childspecs(ChildSpecs, AutoShutdown) -> Result
                          when
                              ChildSpecs :: [child_spec()],
                              AutoShutdown :: undefined | auto_shutdown(),
                              Result :: ok | {error, Error :: term()}.

Takes a list of child specification as argument and returns ok if all of them are syntactically correct, otherwise {error,Error}.

If the AutoShutdown argument is not undefined, also checks if the child specifications are allowed for the given auto_shutdown option.

count_children(SupRef)

(since OTP R13B04)

-spec count_children(SupRef) -> PropListOfCounts
                        when
                            SupRef :: sup_ref(),
                            PropListOfCounts :: [Count],
                            Count ::
                                {specs, ChildSpecCount :: non_neg_integer()} |
                                {active, ActiveProcessCount :: non_neg_integer()} |
                                {supervisors, ChildSupervisorCount :: non_neg_integer()} |
                                {workers, ChildWorkerCount :: non_neg_integer()}.

Returns a property list containing the counts for each of the following elements of the supervisor's child specifications and managed processes:

specs - The total count of children, dead or alive.

active - The count of all actively running child processes managed by this supervisor. For a simple_one_for_one supervisors, no check is done to ensure that each child process is still alive, although the result provided here is likely to be very accurate unless the supervisor is heavily overloaded.

supervisors - The count of all children marked as child_type = supervisor in the specification list, regardless if the child process is still alive.

workers - The count of all children marked as child_type = worker in the specification list, regardless if the child process is still alive.

delete_child(SupRef, Id)

-spec delete_child(SupRef, Id) -> Result
                      when
                          SupRef :: sup_ref(),
                          Id :: child_id(),
                          Result :: ok | {error, Error},
                          Error :: running | restarting | not_found | simple_one_for_one.

Tells supervisor SupRef to delete the child specification identified by Id. The corresponding child process must not be running. Use terminate_child/2 to terminate it.

If successful, the function returns ok. If the child specification identified by Id exists but the corresponding child process is running or is about to be restarted, the function returns {error,running} or {error,restarting}, respectively. If the child specification identified by Id does not exist, the function returns {error,not_found}.

get_childspec(SupRef, Id)

(since OTP 18.0)

-spec get_childspec(SupRef, Id) -> Result
                       when
                           SupRef :: sup_ref(),
                           Id :: pid() | child_id(),
                           Result :: {ok, child_spec()} | {error, Error},
                           Error :: not_found.

Returns the child specification map for the child identified by Id under supervisor SupRef. The returned map contains all keys, both mandatory and optional.

restart_child(SupRef, Id)

-spec restart_child(SupRef, Id) -> Result
                       when
                           SupRef :: sup_ref(),
                           Id :: child_id(),
                           Result ::
                               {ok, Child :: child()} |
                               {ok, Child :: child(), Info :: term()} |
                               {error, Error},
                           Error :: running | restarting | not_found | simple_one_for_one | term().

Tells supervisor SupRef to restart a child process corresponding to the child specification identified by Id. The child specification must exist, and the corresponding child process must not be running.

Notice that for temporary children, the child specification is automatically deleted when the child terminates; thus, it is not possible to restart such children.

If the child specification identified by Id does not exist, the function returns {error,not_found}. If the child specification exists but the corresponding process is already running, the function returns {error,running}.

If the child process start function returns {ok,Child} or {ok,Child,Info}, the pid is added to the supervisor and the function returns the same value.

If the child process start function returns ignore, the pid remains set to undefined and the function returns {ok,undefined}.

If the child process start function returns an error tuple or an erroneous value, or if it fails, the function returns {error,Error}, where Error is a term containing information about the error.

start_child(Supervisor, ChildSpecOrExtraArgs)

-spec start_child(SupRef, ChildSpec) -> startchild_ret()
                     when SupRef :: sup_ref(), ChildSpec :: child_spec();
                 (SupRef, ExtraArgs) -> startchild_ret() when SupRef :: sup_ref(), ExtraArgs :: [term()].

Dynamically adds a child specification to supervisor SupRef, which starts the corresponding child process.

For one_for_one, one_for_all and rest_for_one supervisors, the second argument must be a valid child specification ChildSpec. The child process is started by using the start function as defined in the child specification.

For simple_one_for_one supervisors, the child specification defined in Module:init/1 is used, and the second argument must instead be an arbitrary list of terms ExtraArgs. The child process is then started by appending ExtraArgs to the existing start function arguments, that is, by calling apply(M, F, A++ExtraArgs), where {M,F,A} is the start function defined in the child specification.

If there already exists a child specification with the specified identifier, ChildSpec is discarded, and the function returns {error,already_present} or {error,{already_started,Child}}, depending on if the corresponding child process is running or not.

If the child process start function returns {ok,Child} or {ok,Child,Info}, the child specification and pid are added to the supervisor and the function returns the same value.

If the child process start function returns ignore, the child specification ChildSpec is added to the supervisor if it is an one_for_one, one_for_all or rest_for_one supervisor, and the pid is set to undefined. For simple_one_for_one supervisors, no child is added to the supervisor. The function returns {ok,undefined}.

If the child process start function returns an error tuple or an erroneous value, or if it fails, the child specification is discarded, and the function returns {error,Error}, where Error is a term containing information about the error and child specification.

start_link(Module, Args)

-spec start_link(Module, Args) -> startlink_ret() when Module :: module(), Args :: term().

Creates a nameless supervisor process as part of a supervision tree.

Equivalent to start_link/3 except that the supervisor process is not registered.

start_link(SupName, Module, Args)

-spec start_link(SupName, Module, Args) -> startlink_ret()
                    when SupName :: sup_name(), Module :: module(), Args :: term().

Creates a supervisor process as part of a supervision tree.

For example, the function ensures that the supervisor is linked to the calling process (its supervisor).

The created supervisor process calls Module:init/1 to find out about restart strategy, maximum restart intensity, and child processes. To ensure a synchronized startup procedure, start_link/2,3 does not return until Module:init/1 has returned and all child processes have been started.

If SupName={local,Name}, the supervisor is registered locally as Name using register/2.

If SupName={global,Name}, the supervisor is registered globally as Name using global:register_name/2.

If SupName={via,Module,Name}, the supervisor is registered as Name using the registry represented by Module. The Module callback must export the functions register_name/2, unregister_name/1, and send/2, which must behave like the corresponding functions in global. Thus, {via,global,Name} is a valid reference.

Module is the name of the callback module.

Args is any term that is passed as the argument to Module:init/1.

If the supervisor and its child processes are successfully created (that is, if all child process start functions return {ok,Child}, {ok,Child,Info}, or ignore), the function returns {ok,Pid}, where Pid is the pid of the supervisor.

If there already exists a process with the specified SupName, the function returns {error,{already_started,Pid}}, where Pid is the pid of that process.

If Module:init/1 returns ignore, this function returns ignore as well, and the supervisor terminates with reason normal.

If Module:init/1 fails or returns an incorrect value, this function returns {error,Term}, where Term is a term with information about the error, and the supervisor terminates with reason Term.

If any child process start function fails or returns an error tuple or an erroneous value, the supervisor first terminates all already started child processes with reason shutdown and then terminate itself and returns {error, {shutdown, Reason}}.

stop(SupRef)

(since OTP 29.0)

-spec stop(SupRef :: sup_ref()) -> ok.

Equivalent to stop(SupRef, normal, infinity).

stop(SupRef, Reason, Timeout)

(since OTP 29.0)

-spec stop(SupRef :: sup_ref(), Reason :: term(), Timeout :: timeout()) -> ok.

Stop a supervisor.

Orders the supervisor specified by SupRef to exit with the specified Reason and waits for it to terminate. The supervisor will terminate all its children before exiting.

The function returns ok if the supervisor terminates with the expected reason. Any other reason than normal, shutdown, or {shutdown,Term} causes an error report to be issued using logger. An exit signal with the same reason is sent to linked processes and ports.

Timeout is an integer that specifies how many milliseconds to wait for the supervisor to terminate, or the atom infinity to wait indefinitely. If the supervisor has not terminated within the specified time, the call exits the calling process with reason timeout.

If the process does not exist, the call exits the calling process with reason noproc, or with reason {nodedown,Node} if the connection fails to the remote Node where the supervisor runs.

Warning

Calling this function from a (sub-)child process of the given supervisor will result in a deadlock which will last until either the shutdown timeout of the child or the timeout given to stop/3 has expired.

terminate_child(SupRef, Id)

-spec terminate_child(SupRef, Id) -> Result
                         when
                             SupRef :: sup_ref(),
                             Id :: pid() | child_id(),
                             Result :: ok | {error, Error},
                             Error :: not_found | simple_one_for_one.

Tells supervisor SupRef to terminate the specified child.

If the supervisor is not simple_one_for_one, Id must be the child specification identifier. The process, if any, is terminated and, unless it is a temporary child, the child specification is kept by the supervisor. The child process can later be restarted by the supervisor. The child process can also be restarted explicitly by calling restart_child/2. Use delete_child/2 to remove the child specification.

If the child is temporary, the child specification is deleted as soon as the process terminates. This means that delete_child/2 has no meaning and restart_child/2 cannot be used for these children.

If the supervisor is simple_one_for_one, Id must be the pid/0 of the child process. If the specified process is alive, but is not a child of the specified supervisor, the function returns {error,not_found}. If the child specification identifier is specified instead of a pid/0, the function returns {error,simple_one_for_one}.

If successful, the function returns ok. If there is no child specification with the specified Id, the function returns {error,not_found}.

which_child(SupRef, Id)

(since OTP 28.0)

-spec which_child(SupRef, Id) -> Result
                     when
                         SupRef :: sup_ref(),
                         Id :: pid() | child_id(),
                         Result :: {ok, {Id, Child, Type, Modules}} | {error, Error},
                         Child :: child() | restarting,
                         Type :: worker(),
                         Modules :: modules(),
                         Error :: not_found.

Returns information about the child specification and child process identified by the given Id.

See which_children/1 for an explanation of the information returned.

If no child with the given Id exists, returns {error, not_found}.

which_children(SupRef)

-spec which_children(SupRef) -> [{Id, Child, Type, Modules}]
                        when
                            SupRef :: sup_ref(),
                            Id :: child_id() | undefined,
                            Child :: child() | restarting,
                            Type :: worker(),
                            Modules :: modules().

Returns a newly created list with information about all child specifications and child processes belonging to supervisor SupRef.

Notice that calling this function when supervising many children under low memory conditions can cause an out of memory exception.

The following information is given for each child specification/process:

Id - As defined in the child specification or undefined for a simple_one_for_one supervisor.

Child - The pid of the corresponding child process, the atom restarting if the process is about to be restarted, or undefined if there is no such process.

Type - As defined in the child specification.

Modules - As defined in the child specification.

supervisor_bridge behaviour

Generic supervisor bridge behavior.

This behavior module provides a supervisor bridge, a process that connects a subsystem not designed according to the OTP design principles to a supervision tree. The supervisor bridge sits between a supervisor and the subsystem. It behaves like a real supervisor to its own supervisor, but has a different interface than a real supervisor to the subsystem. For more information, see Supervisor Behaviour in OTP Design Principles.

A supervisor bridge assumes the functions for starting and stopping the subsystem to be located in a callback module exporting a predefined set of functions.

The sys module can be used for debugging a supervisor bridge.

Unless otherwise stated, all functions in this module fail if the specified supervisor bridge does not exist or if bad arguments are specified.

See Also

supervisor, sys

Summary

Callbacks

init(Args)

Whenever a supervisor bridge is started using start_link/2,3, this function is called by the new process to start the subsystem and initialize.

terminate(Reason, State)

This function is called by the supervisor bridge when it is about to terminate. It is to be the opposite of Module:init/1 and stop the subsystem and do any necessary cleaning up. The return value is ignored.

Functions

start_link(Module, Args)

Creates a nameless supervisor bridge process as part of a supervision tree.

start_link(SupBridgeName, Module, Args)

Creates a supervisor bridge process, linked to the calling process, which calls Module:init/1 to start the subsystem.

Callbacks

init(Args)

-callback init(Args :: term()) -> {ok, Pid :: pid(), State :: term()} | ignore | {error, Error :: term()}.

Whenever a supervisor bridge is started using start_link/2,3, this function is called by the new process to start the subsystem and initialize.

Args is the Args argument provided to the start function.

The function is to return {ok,Pid,State}, where Pid is the pid of the main process in the subsystem and State is any term.

If later Pid terminates with a reason Reason, the supervisor bridge terminates with reason Reason as well. If later the supervisor bridge is stopped by its supervisor with reason Reason, it calls Module:terminate(Reason,State) to terminate.

If the initialization fails, the function is to return {error,Error}, where Error is any term, or ignore.

terminate(Reason, State)

-callback terminate(Reason :: shutdown | term(), State :: term()) -> Ignored :: term().

This function is called by the supervisor bridge when it is about to terminate. It is to be the opposite of Module:init/1 and stop the subsystem and do any necessary cleaning up. The return value is ignored.

Reason is shutdown if the supervisor bridge is terminated by its supervisor. If the supervisor bridge terminates because a linked process (apart from the main process of the subsystem) has terminated with reason Term, then Reason becomes Term.

State is taken from the return value of Module:init/1.

Functions

start_link(Module, Args)

-spec start_link(Module, Args) -> Result
                    when
                        Module :: module(),
                        Args :: term(),
                        Result :: {ok, Pid} | ignore | {error, Error},
                        Error :: {already_started, Pid} | term(),
                        Pid :: pid().

Creates a nameless supervisor bridge process as part of a supervision tree.

Equivalent to start_link/3 except that the supervisor process is not registered.

start_link(SupBridgeName, Module, Args)

-spec start_link(SupBridgeName, Module, Args) -> Result
                    when
                        SupBridgeName :: {local, Name} | {global, GlobalName} | {via, Module, ViaName},
                        Name :: atom(),
                        GlobalName :: term(),
                        ViaName :: term(),
                        Module :: module(),
                        Args :: term(),
                        Result :: {ok, Pid} | ignore | {error, Error},
                        Error :: {already_started, Pid} | term(),
                        Pid :: pid().

Creates a supervisor bridge process, linked to the calling process, which calls Module:init/1 to start the subsystem.

To ensure a synchronized startup procedure, this function does not return until Module:init/1 has returned.

If SupBridgeName={local,Name}, the supervisor bridge is registered locally as Name using register/2.

If SupBridgeName={global,GlobalName}, the supervisor bridge is registered globally as GlobalName using global:register_name/2.

If SupBridgeName={via,Module,ViaName}, the supervisor bridge is registered as ViaName using a registry represented by Module. The Module callback is to export functions register_name/2, unregister_name/1, and send/2, which are to behave like the corresponding functions in global. Thus, {via,global,GlobalName} is a valid reference.

Module is the name of the callback module.

Args is an arbitrary term that is passed as the argument to Module:init/1.

If the supervisor bridge and the subsystem are successfully started, the function returns {ok,Pid}, where Pid is is the pid of the supervisor bridge.

If there already exists a process with the specified SupBridgeName, the function returns {error,{already_started,Pid}}, where Pid is the pid of that process.

If Module:init/1 returns ignore, this function returns ignore as well and the supervisor bridge terminates with reason normal.

If Module:init/1 fails or returns an error tuple or an incorrect value, this function returns {error,Error}, where Error is a term with information about the error, and the supervisor bridge terminates with reason Error.

sys behaviour

A functional interface to system messages.

This module contains functions for sending system messages used by programs, and messages used for debugging purposes.

Functions used for implementation of processes are also expected to understand system messages, such as debug messages and code change. These functions must be used to implement the use of system messages for a process; either directly, or through standard behaviors, such as gen_server.

The default time-out is 5000 ms, unless otherwise specified. timeout defines the time to wait for the process to respond to a request. If the process does not respond, the function evaluates exit({timeout, {M, F, A}}).

The functions make references to a debug structure. The debug structure is a list of dbg_opt/0, which is an internal data type used by function handle_system_msg/6. No debugging is performed if it is an empty list.

System Messages

Processes that are not implemented as one of the standard behaviors must still understand system messages. The following three messages must be understood:

Plain system messages. These are received as {system, From, Msg}. The content and meaning of this message are not interpreted by the receiving process module. When a system message is received, function handle_system_msg/6 is called to handle the request.

Shutdown messages. If the process traps exits, it must be able to handle a shutdown request from its parent, the supervisor. The message {'EXIT', Parent, Reason} from the parent is an order to terminate. The process must terminate when this message is received, normally with the same Reason as Parent.

If the modules used to implement the process change dynamically during runtime, the process must understand one more message. An example is the gen_event processes. The message is {_Label, {From, Ref}, get_modules}. The reply to this message is From ! {Ref, Modules}, where Modules is a list of the currently active modules in the process.

This message is used by the release handler to find which processes that execute a certain module. The process can later be suspended and ordered to perform a code change for one of its modules.

System Events

When debugging a process with the functions of this module, the process generates system_events, which are then treated in the debug function. For example, trace formats the system events to the terminal.

Four predefined system events are used when a process receives or sends a message. The process can also define its own system events. It is always up to the process itself to format these events.

Summary

Types

dbg_fun()

dbg_opt()

See the introduction of this manual page.

debug_option()

format_fun()

name()

system_event()

Debug events produced by gen_server, gen_statem and gen_event

Callbacks: Process Implementation Functions

system_code_change(Misc, Module, OldVsn, Extra)

Called from handle_system_msg/6 when the process is to perform a code change. The code change is used when the internal data structure has changed. This function converts argument Misc to the new data structure. OldVsn is attribute vsn of the old version of the Module. If no such attribute is defined, the atom undefined is sent.

system_continue(Parent, Debug, Misc)

Called from handle_system_msg/6 when the process is to continue its execution (for example, after it has been suspended). This function never returns.

system_get_state(Misc)

Called from handle_system_msg/6 when the process is to return a term that reflects its current state. State is the value returned by get_state/2.

system_replace_state(StateFun, Misc)

Called from handle_system_msg/6 when the process is to replace its current state. NState is the value returned by replace_state/3.

system_terminate(Reason, Parent, Debug, Misc)

Called from handle_system_msg/6 when the process is to terminate. For example, this function is called when the process is suspended and its parent orders shutdown. It gives the process a chance to do a cleanup. This function never returns.

Functions

change_code(Name, Module, OldVsn, Extra)

Equivalent to change_code(Name, Module, OldVsn, Extra, 5000).

change_code(Name, Module, OldVsn, Extra, Timeout)

Tells the process to change code.

get_state(Name)

Equivalent to get_state(Name, 5000).

get_state(Name, Timeout)

Gets the state of the process.

get_status(Name)

Equivalent to get_status(Name, 5000).

get_status(Name, Timeout)

Gets the status of the process.

install(Name, FuncSpec)

Equivalent to install(Name, FuncSpec, 5000).

install(Name, FuncSpec, Timeout)

Enables installation of alternative debug functions. An example of such a function is a trigger, a function that waits for some special event and performs some action when the event is generated. For example, turning on low-level tracing.

log(Name, Flag)

Equivalent to log(Name, Flag, 5000).

log(Name, Flag, Timeout)

Turns the logging of system events on or off. If on, a maximum of N events are kept in the debug structure (default is 10).

log_to_file(Name, Flag)

Equivalent to log_to_file(Name, FileName, 5000).

log_to_file(Name, Flag, Timeout)

Enables or disables the logging of all system events in text format to the file. The events are formatted with a function that is defined by the process that generated the event (with a call to handle_debug/4). The file is opened with encoding UTF-8.

no_debug(Name)

Equivalent to no_debug(Name, 5000).

no_debug(Name, Timeout)

Turns off all debugging for the process. This includes functions that are installed explicitly with function install/2,3, for example, triggers.

remove(Name, FuncOrFuncId)

Equivalent to remove(Name, FuncOrFuncId, 5000).

remove(Name, FuncOrFuncId, Timeout)

Removes an installed debug function from the process. Func or FuncId must be the same as previously installed.

replace_state(Name, StateFun)

Equivalent to replace_state(Name, StateFun, 5000).

replace_state(Name, StateFun, Timeout)

Replaces the state of the process, and returns the new state.

resume(Name)

Equivalent to resume(Name, 5000).

resume(Name, Timeout)

Resumes a suspended process.

statistics(Name, Flag)

Equivalent to statistics(Name, Flag, 5000).

statistics(Name, Flag, Timeout)

Enables or disables the collection of statistics. If Flag is get, the statistical collection is returned.

suspend(Name)

Equivalent to suspend(Name, 5000).

suspend(Name, Timeout)

Suspends the process. When the process is suspended, it only responds to other system messages, but not other messages.

terminate(Name, Reason)

Equivalent to terminate(Name, Reason, 5000).

terminate(Name, Reason, Timeout)

Orders the process to terminate with the specified Reason. The termination is done asynchronously, so it is not guaranteed that the process is terminated when the function returns.

trace(Name, Flag)

Equivalent to trace(Name, Flag, 5000).

trace(Name, Flag, Timeout)

Prints all system events on standard_io. The events are formatted with a function that is defined by the process that generated the event (with a call to handle_debug/4).

Process Implementation Functions

debug_options(Options)

Can be used by a process that initiates a debug structure from a list of options. The values of argument Opt are the same as for the corresponding functions.

get_debug(Item, Debug, Default) deprecated

Gets the data associated with a debug option. Default is returned if Item is not found. Can be used by the process to retrieve debug data for printing before it terminates.

get_log(Debug)

Returns the logged system events in the debug structure, that is the last argument to handle_debug/4.

handle_debug(Debug, FormFunc, Extra, Event)

This function is called by a process when it generates a system event. FormFunc is a formatting function, called as FormFunc(Device, Event, Extra) to print the events, which is necessary if tracing is activated. Extra is any extra information that the process needs in the format function, for example, the process name.

handle_system_msg(Msg, From, Parent, Module, Debug, Misc)

This function is used by a process module to take care of system messages. The process receives a {system, From, Msg} message and passes Msg and From to this function.

print_log(Debug)

Prints the logged system events in the debug structure, using FormFunc as defined when the event was generated by a call to handle_debug/4.

Types

dbg_fun()

-type dbg_fun() ::
          fun((FuncState :: _, Event :: system_event(), ProcState :: _) -> done | (NewFuncState :: _)).

dbg_opt()

-opaque dbg_opt()

See the introduction of this manual page.

debug_option()

-type debug_option() ::
          trace | log |
          {log, N :: pos_integer()} |
          statistics |
          {log_to_file, FileName :: file:name()} |
          {install,
           {Func :: dbg_fun(), FuncState :: term()} |
           {FuncId :: term(), Func :: dbg_fun(), FuncState :: term()}}.

format_fun()

(not exported)

-type format_fun() ::
          fun((Device :: io:device() | file:io_device(), Event :: system_event(), Extra :: term()) ->
                  any()).

name()

(not exported)

-type name() :: pid() | atom() | {global, term()} | {via, module(), term()}.

system_event()

-type system_event() ::
          {in, Msg :: _} |
          {in, Msg :: _, State :: _} |
          {out, Msg :: _, To :: _} |
          {out, Msg :: _, To :: _, State :: _} |
          {noreply, State :: _} |
          {continue, Continuation :: _} |
          {postpone, Event :: _, State :: _, NextState :: _} |
          {consume, Event :: _, State :: _, NextState :: _} |
          {start_timer, Action :: _, State :: _} |
          {insert_timeout, Event :: _, State :: _} |
          {enter, Module :: module(), State :: _} |
          {module, Module :: module(), State :: _} |
          {terminate, Reason :: _, State :: _} |
          term().

Debug events produced by gen_server, gen_statem and gen_event

{in,Msg} - Is produced by gen_server and gen_event when the message Msg arrives.

{in,Msg,State} - Is produced by gen_statem when the message Msg arrives in state State.

For gen_statem the Msg term is an {EventType,EventContent} tuple.

{out,Msg,To} - Is produced by gen_statem when the reply Msg is sent back to To by returning a {reply,To,Msg} action from the callback module.

To is of the same type as the first argument to gen_statem:reply/2.

{out,Msg,To,State} - Is produced by gen_server when the reply Msg is sent back to To by returning a {reply,...} tuple from the callback module.

To is of the same type as the first argument to gen_server:reply/2.

State is the new server state.

{noreply,State} - Is produced by gen_server when a {noreply,...} tuple is returned from the callback module.

State is the new server state.

{continue,Continuation} - Is produced by gen_server when a {continue,Continuation} tuple is returned from the callback module.

{postpone,Event,State,NextState} - Is produced by gen_statem when the message Event is postponed in state State. NextState is the new state.

Event is an {EventType,EventContent} tuple.

{consume,Event,State,NextState} - Is produced by gen_statem when the message Event is consumed in state State. NextState is the new state.

Event is an {EventType,EventContent} tuple.

{start_timer,Action,State} - Is produced by gen_statem when the action Action starts a timer in state State.

{insert_timeout,Event,State} - Is produced by gen_statem when a timeout zero action inserts event Event in state State.

Event is an {EventType,EventContent} tuple.

{enter,Module,State} - Is produced by gen_statem when module Module enters the first state State.

{module,Module,State} - Is produced by gen_statem when setting module Module in state State.

{terminate,Reason,State} - Is produced by gen_statem when it terminates with reason Reason in state State.

Callbacks: Process Implementation Functions

system_code_change(Misc, Module, OldVsn, Extra)

-callback system_code_change(Misc, Module, OldVsn, Extra) -> {ok, NMisc}
                                when
                                    Misc :: term(),
                                    OldVsn :: undefined | term(),
                                    Module :: atom(),
                                    Extra :: term(),
                                    NMisc :: term().

Called from handle_system_msg/6 when the process is to perform a code change. The code change is used when the internal data structure has changed. This function converts argument Misc to the new data structure. OldVsn is attribute vsn of the old version of the Module. If no such attribute is defined, the atom undefined is sent.

system_continue(Parent, Debug, Misc)

-callback system_continue(Parent, Debug, Misc) -> no_return()
                             when Parent :: pid(), Debug :: [dbg_opt()], Misc :: term().

Called from handle_system_msg/6 when the process is to continue its execution (for example, after it has been suspended). This function never returns.

system_get_state(Misc)

(since OTP 17.0)

-callback system_get_state(Misc) -> {ok, State} when Misc :: term(), State :: term().

Called from handle_system_msg/6 when the process is to return a term that reflects its current state. State is the value returned by get_state/2.

system_replace_state(StateFun, Misc)

(since OTP 17.0)

-callback system_replace_state(StateFun, Misc) -> {ok, NState, NMisc}
                                  when
                                      Misc :: term(),
                                      NState :: term(),
                                      NMisc :: term(),
                                      StateFun :: fun((State :: term()) -> NState).

Called from handle_system_msg/6 when the process is to replace its current state. NState is the value returned by replace_state/3.

system_terminate(Reason, Parent, Debug, Misc)

-callback system_terminate(Reason, Parent, Debug, Misc) -> no_return()
                              when Reason :: term(), Parent :: pid(), Debug :: [dbg_opt()], Misc :: term().

Called from handle_system_msg/6 when the process is to terminate. For example, this function is called when the process is suspended and its parent orders shutdown. It gives the process a chance to do a cleanup. This function never returns.

Functions

change_code(Name, Module, OldVsn, Extra)

-spec change_code(Name, Module, OldVsn, Extra) -> ok | {error, Reason}
                     when
                         Name :: name(),
                         Module :: module(),
                         OldVsn :: undefined | term(),
                         Extra :: term(),
                         Reason :: term().

Equivalent to change_code(Name, Module, OldVsn, Extra, 5000).

change_code(Name, Module, OldVsn, Extra, Timeout)

-spec change_code(Name, Module, OldVsn, Extra, Timeout) -> ok | {error, Reason}
                     when
                         Name :: name(),
                         Module :: module(),
                         OldVsn :: undefined | term(),
                         Extra :: term(),
                         Timeout :: timeout(),
                         Reason :: term().

Tells the process to change code.

The process must be suspended to handle this message. Argument Extra is reserved for each process to use as its own. Function Module:system_code_change/4 is called. OldVsn is the old version of the Module.

get_state(Name)

(since OTP R16B01)

-spec get_state(Name) -> State when Name :: name(), State :: term().

Equivalent to get_state(Name, 5000).

get_state(Name, Timeout)

(since OTP R16B01)

-spec get_state(Name, Timeout) -> State when Name :: name(), Timeout :: timeout(), State :: term().

Gets the state of the process.

Note

These functions are intended only to help with debugging. They are provided for convenience, allowing developers to avoid having to create their own state extraction functions and also avoid having to interactively extract the state from the return values of get_status/1 or get_status/2 while debugging.

The value of State varies for different types of processes, as follows:

For a gen_server process, the returned State is the state of the callback module.

For a gen_statem process, State is the tuple {CurrentState,CurrentData}.

For a gen_event process, State is a list of tuples, where each tuple corresponds to an event handler registered in the process and contains {Module, Id, HandlerState}, as follows:

Module - The module name of the event handler.

Id - The ID of the handler (which is false if it was registered without an ID).

HandlerState - The state of the handler.

If the callback module exports a function system_get_state/1, it is called in the target process to get its state. Its argument is the same as the Misc value returned by get_status/1,2, and function Module:system_get_state/1 is expected to extract the state of the callback module from it. Function system_get_state/1 must return {ok, State}, where State is the state of the callback module.

If the callback module does not export a system_get_state/1 function, get_state/1,2 assumes that the Misc value is the state of the callback module and returns it directly instead.

If the callback module's system_get_state/1 function crashes or throws an exception, the caller exits with error {callback_failed, {Module, system_get_state}, {Class, Reason}}, where Module is the name of the callback module and Class and Reason indicate details of the exception.

Function system_get_state/1 is primarily useful for user-defined behaviors and modules that implement OTP special processes. The gen_server, gen_statem, and gen_event OTP behavior modules export this function, so callback modules for those behaviors need not to supply their own.

For more information about a process, including its state, see get_status/1 and get_status/2.

get_status(Name)

-spec get_status(Name) -> Status
                    when
                        Name :: name(),
                        Status :: {status, Pid :: pid(), {module, Module :: module()}, [SItem]},
                        SItem ::
                            (PDict :: [{Key :: term(), Value :: term()}]) |
                            (SysState :: running | suspended) |
                            (Parent :: pid()) |
                            (Dbg :: [dbg_opt()]) |
                            (Misc :: term()).

Equivalent to get_status(Name, 5000).

get_status(Name, Timeout)

-spec get_status(Name, Timeout) -> Status
                    when
                        Name :: name(),
                        Timeout :: timeout(),
                        Status :: {status, Pid :: pid(), {module, Module :: module()}, [SItem]},
                        SItem ::
                            (PDict :: [{Key :: term(), Value :: term()}]) |
                            (SysState :: running | suspended) |
                            (Parent :: pid()) |
                            (Dbg :: [dbg_opt()]) |
                            (Misc :: term()).

Gets the status of the process.

The value of Misc varies for different types of processes, for example:

A gen_server process returns the state of the callback module.

A gen_statem process returns information, such as its current state name and state data.

A gen_event process returns information about each of its registered handlers.

A bare sys process returns the value passed as Misc to handle_system_msg/6.

Callback modules for gen_server, gen_statem, and gen_event can also change the value of Misc by exporting a function format_status/1, which contributes module-specific information. For details, see gen_server:format_status/1, gen_statem:format_status/1, and gen_event:format_status/1.

install(Name, FuncSpec)

-spec install(Name, FuncSpec) -> ok
                 when
                     Name :: name(),
                     FuncSpec :: {Func, FuncState} | {FuncId, Func, FuncState},
                     FuncId :: term(),
                     Func :: dbg_fun(),
                     FuncState :: term().

Equivalent to install(Name, FuncSpec, 5000).

install(Name, FuncSpec, Timeout)

-spec install(Name, FuncSpec, Timeout) -> ok
                 when
                     Name :: name(),
                     FuncSpec :: {Func, FuncState} | {FuncId, Func, FuncState},
                     FuncId :: term(),
                     Func :: dbg_fun(),
                     FuncState :: term(),
                     Timeout :: timeout().

Enables installation of alternative debug functions. An example of such a function is a trigger, a function that waits for some special event and performs some action when the event is generated. For example, turning on low-level tracing.

Func is called whenever a system event is generated. This function is to return done, or a new Func state. In the first case, the function is removed. It is also removed if the function fails. If one debug function should be installed more times, a unique FuncId must be specified for each installation.

log(Name, Flag)

-spec log(Name, Flag) -> ok | {ok, [system_event()]}
             when Name :: name(), Flag :: true | {true, N :: pos_integer()} | false | get | print.

Equivalent to log(Name, Flag, 5000).

log(Name, Flag, Timeout)

-spec log(Name, Flag, Timeout) -> ok | {ok, [system_event()]}
             when
                 Name :: name(),
                 Flag :: true | {true, N :: pos_integer()} | false | get | print,
                 Timeout :: timeout().

Turns the logging of system events on or off. If on, a maximum of N events are kept in the debug structure (default is 10).

If Flag is get, a list of all logged events is returned.

If Flag is print, the logged events are printed to standard_io.

The events are formatted with a function that is defined by the process that generated the event (with a call to handle_debug/4).

log_to_file(Name, Flag)

-spec log_to_file(Name, Flag) -> ok | {error, open_file}
                     when Name :: name(), Flag :: (FileName :: string()) | false.

Equivalent to log_to_file(Name, FileName, 5000).

log_to_file(Name, Flag, Timeout)

-spec log_to_file(Name, Flag, Timeout) -> ok | {error, open_file}
                     when Name :: name(), Flag :: (FileName :: string()) | false, Timeout :: timeout().

Enables or disables the logging of all system events in text format to the file. The events are formatted with a function that is defined by the process that generated the event (with a call to handle_debug/4). The file is opened with encoding UTF-8.

no_debug(Name)

-spec no_debug(Name) -> ok when Name :: name().

Equivalent to no_debug(Name, 5000).

no_debug(Name, Timeout)

-spec no_debug(Name, Timeout) -> ok when Name :: name(), Timeout :: timeout().

Turns off all debugging for the process. This includes functions that are installed explicitly with function install/2,3, for example, triggers.

remove(Name, FuncOrFuncId)

-spec remove(Name, Func | FuncId) -> ok when Name :: name(), Func :: dbg_fun(), FuncId :: term().

Equivalent to remove(Name, FuncOrFuncId, 5000).

remove(Name, FuncOrFuncId, Timeout)

-spec remove(Name, Func | FuncId, Timeout) -> ok
                when Name :: name(), Func :: dbg_fun(), FuncId :: term(), Timeout :: timeout().

Removes an installed debug function from the process. Func or FuncId must be the same as previously installed.

replace_state(Name, StateFun)

(since OTP R16B01)

-spec replace_state(Name, StateFun) -> NewState
                       when
                           Name :: name(),
                           StateFun :: fun((State :: term()) -> NewState :: term()),
                           NewState :: term().

Equivalent to replace_state(Name, StateFun, 5000).

replace_state(Name, StateFun, Timeout)

(since OTP R16B01)

-spec replace_state(Name, StateFun, Timeout) -> NewState
                       when
                           Name :: name(),
                           StateFun :: fun((State :: term()) -> NewState :: term()),
                           Timeout :: timeout(),
                           NewState :: term().

Replaces the state of the process, and returns the new state.

Note

These functions are intended only to help with debugging, and are not to be called from normal code. They are provided for convenience, allowing developers to avoid having to create their own custom state replacement functions.

Function StateFun provides a new state for the process. Argument State and the NewState return value of StateFun vary for different types of processes as follows:

For a gen_server process, State is the state of the callback module and NewState is a new instance of that state.

For a gen_statem process, State is the tuple {CurrentState,CurrentData}, and NewState is a similar tuple, which can contain a new current state, new state data, or both.

For a gen_event process, State is the tuple {Module, Id, HandlerState} as follows:

Module - The module name of the event handler.

Id - The ID of the handler (which is false if it was registered without an ID).

HandlerState - The state of the handler.

NewState is a similar tuple where Module and Id are to have the same values as in State, but the value of HandlerState can be different. Returning a NewState, whose Module or Id values differ from those of State, leaves the state of the event handler unchanged. For a gen_event process, StateFun is called once for each event handler registered in the gen_event process.

If a StateFun function decides not to effect any change in process state, then regardless of process type, it can return its State argument.

If a StateFun function crashes or throws an exception, the original state of the process is unchanged for gen_server, and gen_statem processes. For gen_event processes, a crashing or failing StateFun function means that only the state of the particular event handler it was working on when it failed or crashed is unchanged; it can still succeed in changing the states of other event handlers registered in the same gen_event process.

If the callback module exports a system_replace_state/2 function, it is called in the target process to replace its state using StateFun. Its two arguments are StateFun and Misc, where Misc is the same as the Misc value returned by get_status/1,2. A system_replace_state/2 function is expected to return {ok, NewState, NewMisc}, where NewState is the new state of the callback module, obtained by calling StateFun, and NewMisc is a possibly new value used to replace the original Misc (required as Misc often contains the state of the callback module within it).

If the callback module does not export a system_replace_state/2 function, replace_state/2,3 assumes that Misc is the state of the callback module, passes it to StateFun and uses the return value as both the new state and as the new value of Misc.

If the callback module's function system_replace_state/2 crashes or throws an exception, the caller exits with error {callback_failed, {Module, system_replace_state}, {Class, Reason}}, where Module is the name of the callback module and Class and Reason indicate details of the exception. If the callback module does not provide a system_replace_state/2 function and StateFun crashes or throws an exception, the caller exits with error {callback_failed, StateFun, {Class, Reason}}.

Function system_replace_state/2 is primarily useful for user-defined behaviors and modules that implement OTP special processes. The OTP behavior modules gen_server, gen_statem, and gen_event export this function, so callback modules for those behaviors need not to supply their own.

resume(Name)

-spec resume(Name) -> ok when Name :: name().

Equivalent to resume(Name, 5000).

resume(Name, Timeout)

-spec resume(Name, Timeout) -> ok when Name :: name(), Timeout :: timeout().

Resumes a suspended process.

statistics(Name, Flag)

-spec statistics(Name, Flag) -> ok | {ok, Statistics}
                    when
                        Name :: name(),
                        Flag :: true | false | get,
                        Statistics :: [StatisticsTuple] | no_statistics,
                        StatisticsTuple ::
                            {start_time, DateTime1} |
                            {current_time, DateTime2} |
                            {reductions, non_neg_integer()} |
                            {messages_in, non_neg_integer()} |
                            {messages_out, non_neg_integer()},
                        DateTime1 :: file:date_time(),
                        DateTime2 :: file:date_time().

Equivalent to statistics(Name, Flag, 5000).

statistics(Name, Flag, Timeout)

-spec statistics(Name, Flag, Timeout) -> ok | {ok, Statistics}
                    when
                        Name :: name(),
                        Flag :: true | false | get,
                        Statistics :: [StatisticsTuple] | no_statistics,
                        StatisticsTuple ::
                            {start_time, DateTime1} |
                            {current_time, DateTime2} |
                            {reductions, non_neg_integer()} |
                            {messages_in, non_neg_integer()} |
                            {messages_out, non_neg_integer()},
                        DateTime1 :: file:date_time(),
                        DateTime2 :: file:date_time(),
                        Timeout :: timeout().

Enables or disables the collection of statistics. If Flag is get, the statistical collection is returned.

suspend(Name)

-spec suspend(Name) -> ok when Name :: name().

Equivalent to suspend(Name, 5000).

suspend(Name, Timeout)

-spec suspend(Name, Timeout) -> ok when Name :: name(), Timeout :: timeout().

Suspends the process. When the process is suspended, it only responds to other system messages, but not other messages.

terminate(Name, Reason)

(since OTP 18.0)

-spec terminate(Name, Reason) -> ok when Name :: name(), Reason :: term().

Equivalent to terminate(Name, Reason, 5000).

terminate(Name, Reason, Timeout)

(since OTP 18.0)

-spec terminate(Name, Reason, Timeout) -> ok when Name :: name(), Reason :: term(), Timeout :: timeout().

Orders the process to terminate with the specified Reason. The termination is done asynchronously, so it is not guaranteed that the process is terminated when the function returns.

trace(Name, Flag)

-spec trace(Name, Flag) -> ok when Name :: name(), Flag :: boolean().

Equivalent to trace(Name, Flag, 5000).

trace(Name, Flag, Timeout)

-spec trace(Name, Flag, Timeout) -> ok when Name :: name(), Flag :: boolean(), Timeout :: timeout().

Prints all system events on standard_io. The events are formatted with a function that is defined by the process that generated the event (with a call to handle_debug/4).

Process Implementation Functions

debug_options(Options)

-spec debug_options([Opt :: debug_option()]) -> [dbg_opt()].

Can be used by a process that initiates a debug structure from a list of options. The values of argument Opt are the same as for the corresponding functions.

get_debug(Item, Debug, Default)

This function is deprecated. sys:get_debug/3 is deprecated; incorrectly documented and only for internal use. Can often be replaced with sys:get_log/1.

-spec get_debug(Item, Debug, Default) -> term()
                   when Item :: log | statistics, Debug :: [dbg_opt()], Default :: term().

Gets the data associated with a debug option. Default is returned if Item is not found. Can be used by the process to retrieve debug data for printing before it terminates.

get_log(Debug)

(since OTP-22.0)

-spec get_log(Debug) -> [system_event()] when Debug :: [dbg_opt()].

Returns the logged system events in the debug structure, that is the last argument to handle_debug/4.

handle_debug(Debug, FormFunc, Extra, Event)

-spec handle_debug(Debug, FormFunc, Extra, Event) -> [dbg_opt()]
                      when
                          Debug :: [dbg_opt()],
                          FormFunc :: format_fun(),
                          Extra :: term(),
                          Event :: system_event().

This function is called by a process when it generates a system event. FormFunc is a formatting function, called as FormFunc(Device, Event, Extra) to print the events, which is necessary if tracing is activated. Extra is any extra information that the process needs in the format function, for example, the process name.

handle_system_msg(Msg, From, Parent, Module, Debug, Misc)

-spec handle_system_msg(Msg, From, Parent, Module, Debug, Misc) -> no_return()
                           when
                               Msg :: term(),
                               From :: {pid(), Tag :: _},
                               Parent :: pid(),
                               Module :: module(),
                               Debug :: [dbg_opt()],
                               Misc :: term().

This function is used by a process module to take care of system messages. The process receives a {system, From, Msg} message and passes Msg and From to this function.

This function never returns. It calls either of the following functions:

Module:system_continue(Parent, NDebug, Misc), where the process continues the execution.

Module:system_terminate(Reason, Parent, Debug, Misc), if the process is to terminate.

Module must export the following:

system_continue/3

system_terminate/4

system_code_change/4

system_get_state/1

system_replace_state/2

Argument Misc can be used to save internal data in a process, for example, its state. It is sent to Module:system_continue/3 or Module:system_terminate/4.

print_log(Debug)

-spec print_log(Debug) -> ok when Debug :: [dbg_opt()].

Prints the logged system events in the debug structure, using FormFunc as defined when the event was generated by a call to handle_debug/4.

c

Command line interface module.

This module enables users to enter the short form of some commonly used commands.

Note

These functions are intended for interactive use in the Erlang shell only. The module prefix can be omitted.

See Also

filename, compile, erlang, yecc, xref

Summary

Types

cmd_line_arg()

h_return()

hcb_return()

hf_return()

ht_return()

Functions

bt(Pid)

Stack backtrace for a process. Equivalent to erlang:process_display(Pid, backtrace).

c(Module)

Works like c(Module, []).

c(Module, Options)

Compiles and then purges and loads the code for a module. Module can be either a module name or a source file path, with or without .erl extension.

c(Module, Options, Filter)

Compiles and then purges and loads the code for module Module, which must be an atom.

cd(Dir)

Changes working directory to Dir, which can be a relative name, and then prints the name of the new working directory.

erlangrc(PathList)

Search PathList and load .erlang resource file if found.

flush()

Flushes any messages sent to the shell.

h(Module)

Print the documentation for Module

h(Module, Function)

Print the documentation for all Module:Functions (regardless of arity).

h(Module, Function, Arity)

Print the documentation for Module:Function/Arity.

hcb(Module)

Print the callback documentation for Module

hcb(Module, Callback)

Print the callback documentation for all Module:Callbacks (regardless of arity).

hcb(Module, Callback, Arity)

Print the callback documentation for Module:Callback/Arity.

help()

Displays help information: all valid shell internal commands, and commands in this module.

ht(Module)

Print the type documentation for Module

ht(Module, Type)

Print the type documentation for Type in Module regardless of arity.

ht(Module, Type, Arity)

Print the type documentation for Type/Arity in Module.

i()

Equivalent to ni/0.

i(X, Y, Z)

Displays information about a process, Equivalent to process_info(pid(X, Y, Z)), but location transparent.

l(Module)

Purges and loads, or reloads, a module by calling code:purge(Module) followed by code:load_file(Module).

lc(Files)

Compiles a list of files by calling compile:file(File, [report_errors, report_warnings]) for each File in Files.

lm()

Reloads all currently loaded modules that have changed on disk (see mm/0). Returns the list of results from calling l(M) for each such M.

ls()

Lists files in the current directory.

ls(Dir)

Lists files in directory Dir or, if Dir is a file, only lists it.

m()

Displays information about the loaded modules, including the files from which they have been loaded.

m(Module)

Displays information about Module.

memory()

Memory allocation information. Equivalent to erlang:memory/0.

memory(TypeSpec)

Memory allocation information. Equivalent to erlang:memory/1.

mm()

Lists all modified modules. Shorthand for code:modified_modules/0.

nc(File)

Equivalent to nc/2.

nc(File, Options)

Compiles and then loads the code for a file on all nodes. Options defaults to []. Compilation is equivalent to

ni()

i/0 displays system information, listing information about all processes. ni/0 does the same, but for all nodes in the network.

nl(Module)

Loads Module on all nodes.

nregs()

Equivalent to regs/0.

pid(X, Y, Z)

Converts X, Y, Z to pid <X.Y.Z>. This function is only to be used when debugging.

pwd()

Prints the name of the working directory.

q()

This function is shorthand for init:stop(), that is, it causes the node to stop in a controlled fashion.

regs()

regs/0 displays information about all registered processes. nregs/0 does the same, but for all nodes in the network.

uptime()

Prints the node uptime (as specified by erlang:statistics(wall_clock)) in human-readable form.

xm(ModSpec)

Finds undefined functions, unused functions, and calls to deprecated functions in a module by calling xref:m/1.

y(File)

Generates an LALR-1 parser. Equivalent to

y(File, Options)

Generates an LALR-1 parser. Equivalent to

Types

cmd_line_arg()

(not exported)

-type cmd_line_arg() :: atom() | string().

h_return()

(not exported)

-type h_return() :: ok | {error, missing | {unknown_format, unicode:chardata()}}.

hcb_return()

(not exported)

-type hcb_return() :: h_return() | {error, callback_missing}.

hf_return()

(not exported)

-type hf_return() :: h_return() | {error, function_missing}.

ht_return()

(not exported)

-type ht_return() :: h_return() | {error, type_missing}.

Functions

bt(Pid)

-spec bt(Pid) -> ok | undefined when Pid :: pid().

Stack backtrace for a process. Equivalent to erlang:process_display(Pid, backtrace).

c(Module)

-spec c(Module) -> {ok, ModuleName} | error when Module :: file:name(), ModuleName :: module().

Works like c(Module, []).

c(Module, Options)

-spec c(Module, Options) -> {ok, ModuleName} | error
           when
               Module :: file:name(),
               Options :: [compile:option()] | compile:option(),
               ModuleName :: module().

Compiles and then purges and loads the code for a module. Module can be either a module name or a source file path, with or without .erl extension.

If Module is a string, it is assumed to be a source file path, and the compiler will attempt to compile the source file with the options Options. If compilation fails, the old object file (if any) is deleted.

If Module is an atom, a source file with that exact name or with .erl extension will be looked for. If found, the source file is compiled with the options Options. If compilation fails, the old object file (if any) is deleted.

If Module is an atom and is not the path of a source file, then the code path is searched to locate the object file for the module and extract its original compiler options and source path. If the source file is not found in the original location, filelib:find_source/1 is used to search for it relative to the directory of the object file.

The source file is compiled with the the original options appended to the given Options, the output replacing the old object file if and only if compilation succeeds.

Notice that purging the code means that any processes lingering in old code for the module are killed without warning. For more information, see the code module.

c(Module, Options, Filter)

(since OTP 20.0)

-spec c(Module, Options, Filter) -> {ok, ModuleName} | error
           when
               Module :: atom(),
               Options :: [compile:option()],
               Filter :: fun((compile:option()) -> boolean()),
               ModuleName :: module().

Compiles and then purges and loads the code for module Module, which must be an atom.

The code path is searched to locate the object file for module Module and extract its original compiler options and source path. If the source file is not found in the original location, filelib:find_source/1 is used to search for it relative to the directory of the object file.

The source file is compiled with the the original options appended to the given Options, the output replacing the old object file if and only if compilation succeeds. The function Filter specifies which elements to remove from the original compiler options before the new options are added. The Filter fun should return true for options to keep, and false for options to remove.

Notice that purging the code means that any processes lingering in old code for the module are killed without warning. For more information, see the code module.

cd(Dir)

-spec cd(Dir) -> ok when Dir :: file:name().

Changes working directory to Dir, which can be a relative name, and then prints the name of the new working directory.

Example:

2> cd("../erlang").
/home/ron/erlang

erlangrc(PathList)

(since OTP 21.0)

-spec erlangrc(PathList) -> {ok, file:filename()} | {error, term()}
                  when PathList :: [Dir :: file:name()].

Search PathList and load .erlang resource file if found.

flush()

-spec flush() -> ok.

Flushes any messages sent to the shell.

h(Module)

(since OTP 23.0)

-spec h(module()) -> h_return().

Print the documentation for Module

h(Module, Function)

(since OTP 23.0)

-spec h(module(), function()) -> hf_return().

Print the documentation for all Module:Functions (regardless of arity).

h(Module, Function, Arity)

(since OTP 23.0)

-spec h(module(), function(), arity()) -> hf_return().

Print the documentation for Module:Function/Arity.

hcb(Module)

(since OTP 23.0)

-spec hcb(module()) -> h_return().

Print the callback documentation for Module

hcb(Module, Callback)

(since OTP 23.0)

-spec hcb(module(), Callback :: atom()) -> hcb_return().

Print the callback documentation for all Module:Callbacks (regardless of arity).

hcb(Module, Callback, Arity)

(since OTP 23.0)

-spec hcb(module(), Callback :: atom(), arity()) -> hcb_return().

Print the callback documentation for Module:Callback/Arity.

help()

-spec help() -> ok.

Displays help information: all valid shell internal commands, and commands in this module.

ht(Module)

(since OTP 23.0)

-spec ht(module()) -> h_return().

Print the type documentation for Module

ht(Module, Type)

(since OTP 23.0)

-spec ht(module(), Type :: atom()) -> ht_return().

Print the type documentation for Type in Module regardless of arity.

ht(Module, Type, Arity)

(since OTP 23.0)

-spec ht(module(), Type :: atom(), arity()) -> ht_return().

Print the type documentation for Type/Arity in Module.

i()

-spec i() -> ok.

Equivalent to ni/0.

i(X, Y, Z)

-spec i(X, Y, Z) -> [{atom(), term()}]
           when X :: non_neg_integer(), Y :: non_neg_integer(), Z :: non_neg_integer().

Displays information about a process, Equivalent to process_info(pid(X, Y, Z)), but location transparent.

l(Module)

-spec l(Module) -> code:load_ret() when Module :: module().

Purges and loads, or reloads, a module by calling code:purge(Module) followed by code:load_file(Module).

Notice that purging the code means that any processes lingering in old code for the module are killed without warning. For more information, see code/3.

lc(Files)

-spec lc(Files) -> ok | error when Files :: [File :: cmd_line_arg()].

Compiles a list of files by calling compile:file(File, [report_errors, report_warnings]) for each File in Files.

For information about File, see file:filename/0.

lm()

(since OTP 20.0)

-spec lm() -> [code:load_ret()].

Reloads all currently loaded modules that have changed on disk (see mm/0). Returns the list of results from calling l(M) for each such M.

ls()

-spec ls() -> ok.

Lists files in the current directory.

ls(Dir)

-spec ls(Dir) -> ok when Dir :: file:name().

Lists files in directory Dir or, if Dir is a file, only lists it.

m()

-spec m() -> ok.

Displays information about the loaded modules, including the files from which they have been loaded.

m(Module)

-spec m(Module) -> ok when Module :: module().

Displays information about Module.

memory()

-spec memory() -> [{Type, Size}] when Type :: atom(), Size :: non_neg_integer().

Memory allocation information. Equivalent to erlang:memory/0.

memory(TypeSpec)

-spec memory(Type) -> Size when Type :: atom(), Size :: non_neg_integer();
            (Types) -> [{Type, Size}] when Types :: [Type], Type :: atom(), Size :: non_neg_integer().

Memory allocation information. Equivalent to erlang:memory/1.

mm()

(since OTP 20.0)

-spec mm() -> [module()].

Lists all modified modules. Shorthand for code:modified_modules/0.

nc(File)

-spec nc(File) -> {ok, Module} | error when File :: file:name(), Module :: module().

Equivalent to nc/2.

nc(File, Options)

-spec nc(File, Options) -> {ok, Module} | error
            when
                File :: file:name(),
                Options :: [Option] | Option,
                Option :: compile:option(),
                Module :: module().

Compiles and then loads the code for a file on all nodes. Options defaults to []. Compilation is equivalent to:

compile:file(File, Options ++ [report_errors, report_warnings])

ni()

-spec ni() -> ok.

i/0 displays system information, listing information about all processes. ni/0 does the same, but for all nodes in the network.

nl(Module)

-spec nl(Module) -> abcast | error when Module :: module().

Loads Module on all nodes.

nregs()

-spec nregs() -> ok.

Equivalent to regs/0.

pid(X, Y, Z)

-spec pid(X, Y, Z) -> pid() when X :: non_neg_integer(), Y :: non_neg_integer(), Z :: non_neg_integer().

Converts X, Y, Z to pid <X.Y.Z>. This function is only to be used when debugging.

pwd()

-spec pwd() -> ok.

Prints the name of the working directory.

q()

-spec q() -> no_return().

This function is shorthand for init:stop(), that is, it causes the node to stop in a controlled fashion.

regs()

-spec regs() -> ok.

regs/0 displays information about all registered processes. nregs/0 does the same, but for all nodes in the network.

uptime()

(since OTP 18.0)

-spec uptime() -> ok.

Prints the node uptime (as specified by erlang:statistics(wall_clock)) in human-readable form.

xm(ModSpec)

-spec xm(module() | file:filename()) -> XRefMRet :: term().

Finds undefined functions, unused functions, and calls to deprecated functions in a module by calling xref:m/1.

y(File)

-spec y(file:name()) -> YeccFileRet :: term().

Generates an LALR-1 parser. Equivalent to:

yecc:file(File)

For information about File = name(), see filename. For information about YeccRet, see yecc:file/2.

y(File, Options)

-spec y(file:name(), [yecc:option()]) -> YeccFileRet :: yecc:yecc_ret().

Generates an LALR-1 parser. Equivalent to:

yecc:file(File, Options)

For information about File = name(), see filename. For information about Options and YeccRet, see yecc:file/2.

edlin

Line and input interpretter for the erlang shell.

This module reads input, handles any escape sequences that have been configured via edlin_key and outputs action requests. The action requests are handled either by modules group or the user_drv.

Key configuration

You can setup a custom key configuration that overrides the default key configuration. This is done by setting the stdlib application parameter shell_keymap before Erlang is started. If you want to have the same keymap in all Erlang shells you can do so by putting a config file in your user's home directory and then set ERL_FLAGS to load it at startup. For example:

$ cat $HOME/.erlang_keymap.config
[{stdlib,
  [{shell_keymap,
    #{ normal => #{ "\^[A" => clear } }
  }]
}].
$ ERL_FLAGS="-config $HOME/.erlang_keymap" erl

The current keymap configuration can be fetched through edlin:keymap(). If a custom keymap or keymap file is specified, then it will be merged with the default keymap.

The keymap is a map of maps where the keys in the parent map corresponds to different editing modes in the shell. The valid modes currently supported are normal and search.

The keys in the child maps are the escape sequences that are sent from the terminal when a key is pressed and each value is a valid action as seen below.

The default atom is used to specify that an action should happen when a key is pressed that does not have any mapping. Typically used to exit a mode.

See tty - A Command-Line Interface for more information about the default keymap.

Actions

The commands below are the built-in action requests for switching input modes on the normal shell or navigating, or manipulating the line feed. The line feed supports multiple lines.

auto_blink - Automatically close the closest matching opening parenthesis.

backward_char - Move backward one character.

backward_delete_char - Delete the character behind the cursor.

backward_delete_word - Delete the word behind the cursor.

backward_kill_line - Delete all characters from the cursor to the beginning of the line and save them in the kill buffer.

backward_kill_word - Delete the word behind the cursor and save it in the kill buffer.

backward_line - Move backward one line.

backward_word - Move backward one word.

beginning_of_expression - Move to the beginning of the expression.

beginning_of_line - Move to the beginning of the line.

clear - Clear the screen.

clear_line - Clear the current expression.

end_of_expression - Move to the end of the expression.

end_of_line - Move to the end of the line.

forward_char - Move forward one character.

forward_delete_char - Delete the character under the cursor.

forward_line - Move forward one line.

forward_word - Move forward one word.

help - Display help for the module or function closest on the left of the cursor.

help_full - Display the whole help text for the module or function closest on the left of the cursor.

history_down - Move to the next item in the history.

history_up - Move to the previous item in the history.

kill_line - Delete all characters from the cursor to the end of the line and save them in the kill buffer.

kill_word - Delete the word under the cursor and save it in the kill buffer.

move_expand_down - Move down one line in the expand area e.g. help or tab completion pager.

move_expand_up - Move up one line in the expand area e.g. help or tab completion pager.

new_line_finish - Add a newline at the end of the line and try to evaluate the current expression.

newline - Add a newline at the cursor position.

open_editor - Open the current line in an editor e.g. EDITOR="code -w" opens a buffer in vs code. Note that you need to pass a flag to the editor so that it signals the shell when you close the buffer.

redraw_line - Redraw the current line.

scroll_expand_down - Scroll down five lines in the expand area e.g. help or tab completion pager.

scroll_expand_up - Scroll up five lines in the expand area e.g. help or tab completion pager.

search_cancel - Cancel the current search.

search_found - Accept the current search result and submit it.

search_quit - Accept the current search result, but edit it before submitting.

search - Enter search mode, search the history.

skip_down - Skip to the next line in the history that matches the current search expression.

skip_up - Skip to the previous line in the history that matches the current search expression.

tab_expand_full - Output all possible tab completions.

tab_expand_quit - Go back to normal mode.

tab_expand - Autocomplete the current word, or show 5 lines of possible completions.

transpose_char - Swap the character behind the cursor with the one in front of it.

transpose_word - Swap the word behind the cursor with the one in front of it.

yank - Insert the contents of the kill buffer at the cursor position.

Summary

Types

keymap()

A map of maps for each shell mode containing key, action pairs.

Functions

keymap()

Get the current keymap used in the shell. Each key in the parent map represents a shell mode e.g. normal or search. Each map associated with the shell modes contains key sequences represented as strings, paired with an action, which is one of the valid actions mentioned above.

Types

keymap()

(not exported) (since OTP 26.1)

-type keymap() :: #{atom() => #{string() | default => atom()}}.

A map of maps for each shell mode containing key, action pairs.

Functions

keymap()

(since OTP 26.1)

-spec keymap() -> keymap().

Get the current keymap used in the shell. Each key in the parent map represents a shell mode e.g. normal or search. Each map associated with the shell modes contains key sequences represented as strings, paired with an action, which is one of the valid actions mentioned above.

edlin_expand

Shell expansion and formatting of expansion suggestions.

This module provides an expand_fun for the erlang shell expand/1,2. It is possible to override this expand_fun io:setopts/1,2.

Summary

Functions

expand(Bef0)

Equivalent to expand/2.

expand(Bef0, Opts)

The standard expansion function is able to expand strings to valid erlang terms. This includes module names

Functions

expand(Bef0)

(since OTP 26.0)

-spec expand(Bef0) -> {Res, Completion, Matches}
                when
                    Bef0 :: string(),
                    Res :: yes | no,
                    Completion :: string(),
                    Matches :: [Element] | [Section],
                    Element :: {string(), [ElementOption]},
                    ElementOption :: {ending, string()},
                    Section :: #{title := string(), elems := Matches, options := SectionOption},
                    SectionOption ::
                        {highlight_all} |
                        {highlight, string()} |
                        {highlight_param, integer()} |
                        {hide, title} |
                        {hide, result} |
                        {separator, string()}.

Equivalent to expand/2.

expand(Bef0, Opts)

(since OTP 26.0)

-spec expand(Bef0, Opts) -> {Res, Completion, Matches}
                when
                    Bef0 :: string(),
                    Opts :: [Option],
                    Option :: {legacy_output, boolean()},
                    Res :: yes | no,
                    Completion :: string(),
                    Matches :: [Element] | [Section],
                    Element :: {string(), [ElementOption]},
                    ElementOption :: {ending, string()},
                    Section :: #{title := string(), elems := Matches, options := SectionOption},
                    SectionOption ::
                        {highlight_all} |
                        {highlight, string()} |
                        {highlight_param, integer()} |
                        {hide, title} |
                        {hide, result} |
                        {separator, string()}.

The standard expansion function is able to expand strings to valid erlang terms. This includes module names:

1> erla
modules
erlang:

function names:

1> is_ato
functions
is_atom(
2> erlang:is_ato
functions
is_atom(

function types:

1> erlang:is_atom(
typespecs
erlang:is_atom(Term)
any()

and automatically add , or closing parenthesis when no other valid expansion is possible. The expand function also completes: shell bindings, record names, record fields and map keys.

As seen below, function headers are grouped together if they've got the same expansion suggestion, in this case all had the same suggestions, that is '}'. There is also limited support for filtering out function typespecs that that does not match the types on the terms on the prompt. Only 4 suggestions are shown below but there exists plenty more typespecs for erlang:system_info.

1> erlang:system_info({allocator, my_allocator
typespecs
erlang:system_info(wordsize | {wordsize, ...} | {wordsize, ...})
erlang:system_info({allocator, ...})
erlang:system_info({allocator_sizes, ...})
erlang:system_info({cpu_topology, ...})
}

The return type of expand function specifies either a list of Element tuples or a list of Section maps. The section concept was introduced to enable more formatting options for the expansion results. For example, the shell expansion has support to highlight text and hide suggestions. There are also a {highlight, Text} that highlights all occurances of Text in the title, and a highlight_all for simplicity which highlights the whole title, as can be seen above for functions and typespecs.

By setting the {hide, result} or {hide, title} options you may hide suggestions. Sometimes the title isn't useful and just produces text noise, in the example above the any/0 result is part of a section with title Types. Hiding results is currently not in use, but the idea is that a section can be selected in the expand area and all the other section entries should be collapsed.

Its possible to set a custom separator between the title and the results. This can be done with {separator, Separator}. By default its set to be \n, some results display a type_name() ::followed by all types that define type_name().

The {ending, Text} ElementOption just appends Text to the Element.

shell

The Erlang shell.

The shell is a user interface program for entering expression sequences. The expressions are evaluated and a value is returned. The shell provides an Emacs like set of shortcuts for editing the text of the current line. See tty - A Command-Line Interface in the ERTS User's Guide for a list of all available shortcuts. You may also change the shortcuts to suit your preferences more, see edlin - line editor in the shell.

A history mechanism saves previous commands and their values, which can then be incorporated in later commands. How many commands and results to save can be determined by the user, either interactively, by calling history/1 and results/1, or by setting the application configuration parameters shell_history_length and shell_saved_results for the STDLIB application. The shell history can be saved to disk by setting the application configuration parameter shell_history for the Kernel application.

The shell uses a helper process for evaluating commands to protect the history mechanism from exceptions. By default the evaluator process is killed when an exception occurs, but by calling catch_exception/1 or by setting the application configuration parameter shell_catch_exception for the STDLIB application this behavior can be changed. See also the example below.

Variable bindings, and local process dictionary changes that are generated in user expressions are preserved, and the variables can be used in later commands to access their values. The bindings can also be forgotten so the variables can be reused.

The special shell commands all have the syntax of (local) function calls. They are evaluated as normal function calls and many commands can be used in one expression sequence.

If a command (local function call) is not recognized by the shell, an attempt is first made to find the function in module user_default, where customized local commands can be placed. If found, the function is evaluated, otherwise an attempt is made to evaluate the function in module shell_default. Module user_default must be explicitly loaded.

The shell also permits the user to start multiple concurrent jobs. A job can be regarded as a set of processes that can communicate with the shell.

There is some support for reading and printing records in the shell. During compilation record expressions are translated to tuple expressions. In runtime it is not known whether a tuple represents a record, and the record definitions used by the compiler are unavailable at runtime. So, to read the record syntax and print tuples as records when possible, record definitions must be maintained by the shell itself.

The shell commands for reading, defining, forgetting, listing, and printing records are described below. Notice that each job has its own set of record definitions. To facilitate matters, record definitions in modules shell_default and user_default (if loaded) are read each time a new job is started. For example, adding the following line to user_default makes the definition of file_info readily available in the shell:

-include_lib("kernel/include/file.hrl").

The shell runs in two modes:

Normal (possibly restricted) mode, in which commands can be edited and expressions evaluated

Job Control Mode, JCL, in which jobs can be started, killed, detached, and connected

Only the currently connected job can 'talk' to the shell.

Shell Commands

The commands below are the built-in shell commands that are always available. In most system the commands listed in the c module are also available in the shell.

b() - Prints the current variable bindings.

f() - Removes all variable bindings.

f(X) - Removes the binding of variable X.

Note

If a huge value is stored in a variable binding, you have to both call f(X) and call history(0) or results(0) to free up that memory.

h() - Prints the history list.

history(N) - Sets the number of previous commands to keep in the history list to N. The previous number is returned. Defaults to 20.

results(N) - Sets the number of results from previous commands to keep in the history list to N. The previous number is returned. Defaults to 20.

e(N) - Repeats command N, if N is positive. If it is negative, the Nth previous command is repeated (that is, e(-1) repeats the previous command).

v(N) - Uses the return value of command N in the current command, if N is positive. If it is negative, the return value of the Nth previous command is used (that is, v(-1) uses the value of the previous command).

help() - Evaluates shell_default:help().

h(Module, Function) - Print the documentation for Module:Function in the shell if available.

ht(Module, Type) - Print the documentation for Module:Type in the shell if available.

hcb(Module, Callback) - Print the documentation for Module:Callback in the shell if available.

c(Mod) - Evaluates shell_default:c(Mod). This compiles and loads the module Mod and purges old versions of the code, if necessary. Mod can be either a module name or a a source file path, with or without .erl extension.

catch_exception(Bool) - Sets the exception handling of the evaluator process. The previous exception handling is returned. The default (false) is to kill the evaluator process when an exception occurs, which causes the shell to create a new evaluator process. When the exception handling is set to true, the evaluator process lives on. This means, for example, that ports and ETS tables as well as processes linked to the evaluator process survive the exception.

rd(RecordName, RecordDefinition) - Defines a record in the shell. RecordName is an atom and RecordDefinition lists the field names and the default values. Usually record definitions are made known to the shell by use of the rr/1,2,3 commands described below, but sometimes it is handy to define records on the fly.

rf() - Removes all record definitions, then reads record definitions from the modules shell_default and user_default (if loaded). Returns the names of the records defined.

rf(RecordNames) - Removes selected record definitions. RecordNames is a record name or a list of record names. To remove all record definitions, use '_'.

rl() - Prints all record definitions.

rl(RecordNames) - Prints selected record definitions. RecordNames is a record name or a list of record names.

rp(Term) - Prints a term using the record definitions known to the shell. All of Term is printed; the depth is not limited as is the case when a return value is printed.

rr(Module) - Reads record definitions from a module's BEAM file. If there are no record definitions in the BEAM file, the source file is located and read instead. Returns the names of the record definitions read. Module is an atom.

rr(Wildcard) - Reads record definitions from files. Existing definitions of any of the record names read are replaced. Wildcard is a wildcard string as defined in filelib, but not an atom.

rr(WildcardOrModule, RecordNames) - Reads record definitions from files but discards record names not mentioned in RecordNames (a record name or a list of record names).

rr(WildcardOrModule, RecordNames, Options) - Reads record definitions from files. The compiler options {i, Dir}, {d, Macro}, and {d, Macro, Value} are recognized and used for setting up the include path and macro definitions. To read all record definitions, use '_' as value of RecordNames.

lf() - Outputs locally defined function with function specs if they exist.

lt() - Outputs locally defined types.

lr() - Outputs locally defined records.

ff() - Forget locally defined functions (including function specs if they exist).

ff({FunName,Arity}) - Forget a locally defined function (including function spec if it exist). Where FunName is the name of the function as an atom and Arity is an integer.

tf() - Forget locally defined types.

tf(Type) - Forget locally defined type where Type is the name of the type represented as an atom.

fl() - Forget locally defined functions, types and records.

save_module(FilePath) - Saves all locally defined functions, types and records to a module file, where FilePath should include both the path to the file and the name of the module with .erl suffix.

Example: src/my_module.erl

Example

The following example is a long dialog with the shell. Commands starting with > are inputs to the shell. All other lines are output from the shell.

strider 1> erl
Erlang (BEAM) emulator version 5.3 [hipe] [threads:0]

Eshell V5.3  (abort with ^G)
1> Str = "abcd".
"abcd"

Command 1 sets variable Str to string "abcd".

2> L = length(Str).
4

Command 2 sets L to the length of string Str.

3> Descriptor = {L, list_to_atom(Str)}.
{4,abcd}

Command 3 builds the tuple Descriptor, evaluating the BIF list_to_atom/1 .

4> L.
4

Command 4 prints the value of variable L.

5> b().
Descriptor = {4,abcd}
L = 4
Str = "abcd"
ok

Command 5 evaluates the internal shell command b(), which is an abbreviation of "bindings". This prints the current shell variables and their bindings. ok at the end is the return value of function b().

6> f(L).
ok

Command 6 evaluates the internal shell command f(L) (abbreviation of "forget"). The value of variable L is removed.

7> b().
Descriptor = {4,abcd}
Str = "abcd"
ok

Command 7 prints the new bindings.

8> f(L).
ok

Command 8 has no effect, as L has no value.

9> {L, _} = Descriptor.
{4,abcd}

Command 9 performs a pattern matching operation on Descriptor, binding a new value to L.

10> L.
4

Command 10 prints the current value of L.

11> {P, Q, R} = Descriptor.
** exception error: no match of right hand side value {4,abcd}

Command 11 tries to match {P, Q, R} against Descriptor, which is {4, abc}. The match fails and none of the new variables become bound. The printout starting with "** exception error:" is not the value of the expression (the expression had no value because its evaluation failed), but a warning printed by the system to inform the user that an error has occurred. The values of the other variables (L, Str, and so on) are unchanged.

12> P.
* 1:1: variable 'P' is unbound
13> Descriptor.
{4,abcd}

Commands 12 and 13 show that P is unbound because the previous command failed, and that Descriptor has not changed.

14>{P, Q} = Descriptor.
{4,abcd}
15> P.
4

Commands 14 and 15 show a correct match where P and Q are bound.

16> f().
ok

Command 16 clears all bindings.

The next few commands assume that test1:demo(X) is defined as follows:

demo(X) ->
    put(aa, worked),
    X = 1,
    X + 10.

17> put(aa, hello).
undefined
18> get(aa).
hello

Commands 17 and 18 set and inspect the value of item aa in the process dictionary.

19> Y = test1:demo(1).
11

Command 19 evaluates test1:demo(1). The evaluation succeeds and the changes made in the process dictionary become visible to the shell. The new value of dictionary item aa can be seen in command 20.

20> get().
[{aa,worked}]
21> put(aa, hello).
worked
22> Z = test1:demo(2).
** exception error: no match of right hand side value 1
     in function  test1:demo/1

Commands 21 and 22 change the value of dictionary item aa to hello and call test1:demo(2). Evaluation fails and the changes made to the dictionary in test1:demo(2), before the error occurred, are discarded.

23> Z.
* 1:1: variable 'Z' is unbound
24> get(aa).
hello

Commands 23 and 24 show that Z was not bound and that dictionary item aa has retained its original value.

25> erase(), put(aa, hello).
undefined
26> spawn(test1, demo, [1]).
<0.57.0>
27> get(aa).
hello

Commands 25, 26, and 27 show the effect of evaluating test1:demo(1) in the background. In this case, the expression is evaluated in a newly spawned process. Any changes made in the process dictionary are local to the newly spawned process and therefore not visible to the shell.

28> io:format("hello hello\n").
hello hello
ok
29> e(28).
hello hello
ok
30> v(28).
ok

Commands 28, 29 and 30 use the history facilities of the shell. Command 29 re-evaluates command 28. Command 30 uses the value (result) of command 28. In the cases of a pure function (a function with no side effects), the result is the same. For a function with side effects, the result can be different.

The next few commands show some record manipulation. It is assumed that ex.erl defines a record as follows:

-record(rec, {a, b = val()}).

val() ->
    3.

31> c(ex).
{ok,ex}
32> rr(ex).
[rec]

Commands 31 and 32 compile file ex.erl and read the record definitions in ex.beam. If the compiler did not output any record definitions on the BEAM file, rr(ex) tries to read record definitions from the source file instead.

33> rl(rec).
-record(rec,{a,b = val()}).
ok

Command 33 prints the definition of the record named rec.

34> #rec{}.
** exception error: undefined shell command val/0

Command 34 tries to create a rec record, but fails as function val/0 is undefined.

35> #rec{b = 3}.
#rec{a = undefined,b = 3}

Command 35 shows the workaround: explicitly assign values to record fields that cannot otherwise be initialized.

36> rp(v(-1)).
#rec{a = undefined,b = 3}
ok

Command 36 prints the newly created record using record definitions maintained by the shell.

37> rd(rec, {f = orddict:new()}).
rec

Command 37 defines a record directly in the shell. The definition replaces the one read from file ex.beam.

38> #rec{}.
#rec{f = []}
ok

Command 38 creates a record using the new definition, and prints the result.

39> rd(rec, {c}), A.
* 1:15: variable 'A' is unbound
40> #rec{}.
#rec{c = undefined}
ok

Command 39 and 40 show that record definitions are updated as side effects. The evaluation of the command fails, but the definition of rec has been carried out.

For the next command, it is assumed that test1:loop(N) is defined as follows:

loop(N) ->
    io:format("Hello Number: ~w~n", [N]),
    loop(N+1).

41> test1:loop(0).
Hello Number: 0
Hello Number: 1
Hello Number: 2
Hello Number: 3

User switch command
 --> i
 --> c
.
.
.
Hello Number: 3374
Hello Number: 3375
Hello Number: 3376
Hello Number: 3377
Hello Number: 3378
** exception exit: killed

Command 41 evaluates test1:loop(0), which puts the system into an infinite loop. At this point the user types ^G (Control G), which suspends output from the current process, which is stuck in a loop, and activates JCL mode. In JCL mode the user can start and stop jobs.

In this particular case, command i ("interrupt") terminates the looping program, and command c connects to the shell again. As the process was running in the background before we killed it, more printouts occur before message "** exception exit: killed" is shown.

42> E = ets:new(t, []).
#Ref<0.1662103692.2407923716.214192>

Command 42 creates an ETS table.

43> ets:insert({d,1,2}).
** exception error: undefined function ets:insert/1

Command 43 tries to insert a tuple into the ETS table, but the first argument (the table) is missing. The exception kills the evaluator process.

44> ets:insert(E, {d,1,2}).
** exception error: argument is of wrong type
     in function  ets:insert/2
        called as ets:insert(16,{d,1,2})

Command 44 corrects the mistake, but the ETS table has been destroyed as it was owned by the killed evaluator process.

45> f(E).
ok
46> catch_exception(true).
false

Command 46 sets the exception handling of the evaluator process to true. The exception handling can also be set when starting Erlang by erl -stdlib shell_catch_exception true.

47> E = ets:new(t, []).
#Ref<0.1662103692.2407923716.214197>
48> ets:insert({d,1,2}).
* exception error: undefined function ets:insert/1

Command 48 makes the same mistake as in command 43, but this time the evaluator process lives on. The single star at the beginning of the printout signals that the exception has been caught.

49> ets:insert(E, {d,1,2}).
true

Command 49 successfully inserts the tuple into the ETS table.

50> ets:insert(#Ref<0.1662103692.2407923716.214197>, {e,3,4}).
true

Command 50 inserts another tuple into the ETS table. This time the first argument is the table identifier itself. The shell can parse commands with pids (<0.60.0>), ports (#Port<0.536>), references (#Ref<0.1662103692.2407792644.214210>), and external functions (#Fun<a.b.1>), but the command fails unless the corresponding pid, port, reference, or function can be created in the running system.

51> halt().
strider 2>

Command 51 exits the Erlang runtime system.

JCL Mode

When the shell starts, it starts a single evaluator process. This process, together with any local processes that it spawns, is referred to as a job. Only the current job, which is said to be connected, can perform operations with standard I/O. All other jobs, which are said to be detached, are blocked if they attempt to use standard I/O.

All jobs that do not use standard I/O run in the normal way.

The shell escape key ^G (Control G) detaches the current job and activates JCL mode. The JCL mode prompt is "-->". If "?" is entered at the prompt, the following help message is displayed:

--> ?
c [nn]            - connect to job
i [nn]            - interrupt job
k [nn]            - kill job
j                 - list all jobs
s [shell]         - start local shell
r [node [shell]]  - start remote shell
q                 - quit erlang
? | h             - this message

The JCL commands have the following meaning:

c [nn] - Connects to job number <nn> or the current job. The standard shell is resumed. Operations that use standard I/O by the current job are interleaved with user inputs to the shell.

i [nn] - Stops the current evaluator process for job number nn or the current job, but does not kill the shell process. So, any variable bindings and the process dictionary are preserved and the job can be connected again. This command can be used to interrupt an endless loop.

k [nn] - Kills job number nn or the current job. All spawned processes in the job are killed, provided they have not evaluated the group_leader/1 BIF and are located on the local machine. Processes spawned on remote nodes are not killed.

j - Lists all jobs. A list of all known jobs is printed. The current job name is prefixed with '*'.

s - Starts a new job. This is assigned the new index [nn], which can be used in references.

s [shell] - Starts a new job. This is assigned the new index [nn], which can be used in references. If optional argument shell is specified, it is assumed to be a module that implements an alternative shell.

r [node] - Starts a remote job on node. This is used in distributed Erlang to allow a shell running on one node to control a number of applications running on a network of nodes. If optional argument shell is specified, it is assumed to be a module that implements an alternative shell.

q - Quits Erlang. Notice that this option is disabled if Erlang is started with the ignore break, +Bi, system flag (which can be useful, for example when running a restricted shell, see the next section).

? - Displays the help message above.

The behavior of shell escape can be changed by the STDLIB application variable shell_esc. The value of the variable can be either jcl (erl -stdlib shell_esc jcl) or abort (erl -stdlib shell_esc abort). The first option sets ^G to activate JCL mode (which is also default behavior). The latter sets ^G to terminate the current shell and start a new one. JCL mode cannot be invoked when shell_esc is set to abort.

If you want an Erlang node to have a remote job active from the start (rather than the default local job), start Erlang with flag -remsh, for example, erl -remsh other_node@other_host

Restricted Shell

The shell can be started in a restricted mode. In this mode, the shell evaluates a function call only if allowed. This feature makes it possible to, for example, prevent a user from accidentally calling a function from the prompt that could harm a running system (useful in combination with system flag +Bi).

When the restricted shell evaluates an expression and encounters a function call or an operator application, it calls a callback function (with information about the function call in question). This callback function returns true to let the shell go ahead with the evaluation, or false to abort it. There are two possible callback functions for the user to implement:

local_allowed(Func, ArgList, State) -> {boolean(),NewState}

This is used to determine if the call to the local function Func with arguments ArgList is to be allowed.

non_local_allowed(FuncSpec, ArgList, State) -> {boolean(),NewState} | {{redirect,NewFuncSpec,NewArgList},NewState}

This is used to determine if the call to non-local function FuncSpec ({Module,Func} or a fun) with arguments ArgList is to be allowed. The return value {redirect,NewFuncSpec,NewArgList} can be used to let the shell evaluate some other function than the one specified by FuncSpec and ArgList.

These callback functions are called from local and non-local evaluation function handlers, described in the erl_eval manual page. (Arguments in ArgList are evaluated before the callback functions are called.)

From OTP 25.0, if there are errors evaluating Erlang constructs, such as badmatch during pattern matching or bad_generator in a comprehension, the evaluator will dispatch to erlang:raise(error, Reason, Stacktrace). This call will be checked against the non_local_allowed/3 callback function. You can either forbid it, allow it, or redirect to another call of your choice.

Argument State is a tuple {ShellState,ExprState}. The return value NewState has the same form. This can be used to carry a state between calls to the callback functions. Data saved in ShellState lives through an entire shell session. Data saved in ExprState lives only through the evaluation of the current expression.

There are two ways to start a restricted shell session:

Use STDLIB application variable restricted_shell and specify, as its value, the name of the callback module. Example (with callback functions implemented in callback_mod.erl): $ erl -stdlib restricted_shell callback_mod.

From a normal shell session, call function start_restricted/1. This exits the current evaluator and starts a new one in restricted mode.

Notes:

When restricted shell mode is activated or deactivated, new jobs started on the node run in restricted or normal mode, respectively.

If restricted mode has been enabled on a particular node, remote shells connecting to this node also run in restricted mode.

The callback functions cannot be used to allow or disallow execution of functions called from compiled code (only functions called from expressions entered at the shell prompt).

Errors when loading the callback module is handled in different ways depending on how the restricted shell is activated:

If the restricted shell is activated by setting the STDLIB variable during emulator startup, and the callback module cannot be loaded, a default restricted shell allowing only the commands q() and init:stop() is used as fallback.

If the restricted shell is activated using start_restricted/1 and the callback module cannot be loaded, an error report is sent to the error logger and the call returns {error,Reason}.

Prompting

The default shell prompt function displays the name of the node (if the node can be part of a distributed system) and the current command number. The user can customize the prompt function by calling prompt_func/1 or by setting application configuration parameter shell_prompt_func for the STDLIB application. Similarly the multiline prompt can be configured as well, by calling multiline_prompt_func/1 or by setting the application parameter shell_multiline_prompt for the STDLIB application.

A customized prompt function is stated as a tuple {Mod, Func}. The function is called as Mod:Func(L), where L is a list of key-value pairs created by the shell. Currently there is only one pair: {history, N}, where N is the current command number. The function is to return a list of characters or an atom. This constraint is because of the Erlang I/O protocol. Unicode characters beyond code point 255 are allowed in the list and the atom. Notice that in restricted mode the call Mod:Func(L) must be allowed or the default shell prompt function is called.

Summary

Functions

catch_exception(Bool)

Sets the exception handling of the evaluator process. The previous exception handling is returned. The default (false) is to kill the evaluator process when an exception occurs, which causes the shell to create a new evaluator process. When the exception handling is set to true, the evaluator process lives on, which means that, for example, ports and ETS tables as well as processes linked to the evaluator process survive the exception.

default_multiline_prompt(Pbs)

Configures the multiline prompt as two trailing dots. This is the default function but it may also be set explicitly as -stdlib shell_multiline_prompt {shell, default_multiline_prompt}.

erl_pp_format_func(String)

A formatting function that can be set with format_shell_func/1 that will make expressions submitted to the shell prettier.

format_shell_func(ShellFormatFunc)

Can be used to set the formatting of the Erlang shell output.

hints(Hints)

Sets printing of shell hints. The previous value of the flag is returned.

history(N)

Sets the number of previous commands to keep in the history list to N. The previous number is returned. Defaults to 20.

inverted_space_prompt(Pbs)

Configures the multiline prompt as inverted space. It may be set explicitly as -stdlib shell_multiline_prompt {shell, inverted_space_prompt} or calling multiline_prompt_func({shell, inverted_space_prompt}).

multiline_prompt_func(PromptFunc)

Sets the shell multiline prompt function to PromptFunc. The previous prompt function is returned.

prompt_func(PromptFunc)

Sets the shell prompt function to PromptFunc. The previous prompt function is returned.

prompt_width(String)

Equivalent to prompt_width/2 with Encoding set to the encoding used by io:user/0.

prompt_width(String, Encoding)

It receives a prompt and computes its width, considering its Unicode characters and ANSI escapes.

results(N)

Sets the number of results from previous commands to keep in the history list to N. The previous number is returned. Defaults to 20.

start_interactive()

Starts the interactive shell if it has not already been started. It can be used to programatically start the shell from an escript or when erl is started with the -noinput or -noshell flags.

start_interactive/1

Starts the interactive shell if it has not already been started. It can be used to programatically start the shell from an escript or when erl is started with the -noinput or -noshell flags. The following options are allowed

start_restricted(Module)

Exits a normal shell and starts a restricted shell. Module specifies the callback module for the functions local_allowed/3 and non_local_allowed/3. The function is meant to be called from the shell.

stop_restricted()

Exits a restricted shell and starts a normal shell. The function is meant to be called from the shell.

strings(Strings)

Sets pretty printing of lists to Strings. The previous value of the flag is returned.

whereis()

Returns the current shell process on the node where the calling process' group_leader is located. If that node has no shell this function will return undefined.

Functions

catch_exception(Bool)

-spec catch_exception(Bool) -> boolean() when Bool :: boolean().

Sets the exception handling of the evaluator process. The previous exception handling is returned. The default (false) is to kill the evaluator process when an exception occurs, which causes the shell to create a new evaluator process. When the exception handling is set to true, the evaluator process lives on, which means that, for example, ports and ETS tables as well as processes linked to the evaluator process survive the exception.

default_multiline_prompt(Pbs)

(since OTP 27.0)

-spec default_multiline_prompt(unicode:chardata()) -> unicode:chardata().

Configures the multiline prompt as two trailing dots. This is the default function but it may also be set explicitly as -stdlib shell_multiline_prompt {shell, default_multiline_prompt}.

erl_pp_format_func(String)

(since OTP 27.0)

-spec erl_pp_format_func(String) -> String2 when String :: string(), String2 :: string().

A formatting function that can be set with format_shell_func/1 that will make expressions submitted to the shell prettier.

Note

This formatting function filter comments away from the expressions.

format_shell_func(ShellFormatFunc)

(since OTP 27.0)

-spec format_shell_func(ShellFormatFunc) -> ShellFormatFunc2
                           when
                               ShellFormatFunc :: default | {module(), function()} | string(),
                               ShellFormatFunc2 :: default | {module(), function()} | string().

Can be used to set the formatting of the Erlang shell output.

This has an effect on commands that have been submitted, and how it is saved in history. Or if the formatting hotkey is pressed while editing an expression (Alt+R by default). You can specify a Mod:Func/1 that expects the whole expression as a string and returns a formatted expressions as a string. See stdlib app config for how to set it before shell started.

If instead a string is provided, it will be used as a shell command. Your command must include ${file} somewhere in the string, for the shell to know where the file goes in the command.

shell:format_shell_func("\"emacs -batch \${file} -l ~/erlang-format/emacs-format-file -f emacs-format-function\"").

shell:format_shell_func({shell, erl_pp_format_func}).

hints(Hints)

(since OTP 28.1)

-spec hints(Hints) -> OldHints when Hints :: boolean(), OldHints :: boolean().

Sets printing of shell hints. The previous value of the flag is returned.

The flag can also be set by the STDLIB application variable shell_hints. Defaults to true, which means that hints will be printed by default. Value false means that no hints are printed in the shell.

history(N)

-spec history(N) -> non_neg_integer() when N :: non_neg_integer().

Sets the number of previous commands to keep in the history list to N. The previous number is returned. Defaults to 20.

inverted_space_prompt(Pbs)

(since OTP 27.0)

-spec inverted_space_prompt(unicode:chardata()) -> unicode:chardata().

Configures the multiline prompt as inverted space. It may be set explicitly as -stdlib shell_multiline_prompt {shell, inverted_space_prompt} or calling multiline_prompt_func({shell, inverted_space_prompt}).

multiline_prompt_func(PromptFunc)

(since OTP 27.0)

-spec multiline_prompt_func(PromptFunc) -> PromptFunc2
                               when
                                   PromptFunc :: default | {module(), function()} | string(),
                                   PromptFunc2 :: default | {module(), function()} | string().

Sets the shell multiline prompt function to PromptFunc. The previous prompt function is returned.

prompt_func(PromptFunc)

(since OTP R13B04)

-spec prompt_func(PromptFunc) -> PromptFunc2
                     when
                         PromptFunc :: default | {module(), atom()},
                         PromptFunc2 :: default | {module(), atom()}.

Sets the shell prompt function to PromptFunc. The previous prompt function is returned.

prompt_width(String)

(since OTP 27.0)

-spec prompt_width(unicode:chardata()) -> non_neg_integer().

Equivalent to prompt_width/2 with Encoding set to the encoding used by io:user/0.

prompt_width(String, Encoding)

(since OTP 27.0)

-spec prompt_width(unicode:chardata(), unicode | latin1) -> non_neg_integer().

It receives a prompt and computes its width, considering its Unicode characters and ANSI escapes.

Useful for creating custom multiline prompts.

Example:

1> shell:prompt_width("olá> ", unicode).
5
%% "olá> " is printed as "ol\341> " on a latin1 systems
2> shell:prompt_width("olá> ", latin1).
8
%% Ansi escapes are ignored
3> shell:prompt_width("\e[32molá\e[0m> ", unicode).
5
%% Double width characters count as 2
4> shell:prompt_width("😀> ", unicode).
4
%% "😀> " is printed as "\x{1F600}> " on latin1 systems
5> shell:prompt_width("😀> ", latin1).
11

results(N)

-spec results(N) -> non_neg_integer() when N :: non_neg_integer().

Sets the number of results from previous commands to keep in the history list to N. The previous number is returned. Defaults to 20.

start_interactive()

(since OTP 26.0)

-spec start_interactive() -> ok | {error, already_started}.

Starts the interactive shell if it has not already been started. It can be used to programatically start the shell from an escript or when erl is started with the -noinput or -noshell flags.

Calling this function will start a remote shell if -remsh is given on the command line or a local shell if not.

start_interactive/1

(since OTP 26.0)

-spec start_interactive(noshell | {noshell, raw | cooked} | {module(), atom(), [term()]}) ->
                           ok | {error, already_started};
                       ({remote, string()}) -> ok | {error, already_started | noconnection};
                       ({node(), {module(), atom(), [term()]}} |
                        {remote, string(), {module(), atom(), [term()]}}) ->
                           ok |
                           {error, already_started | noconnection | badfile | nofile | on_load_failure}.

Starts the interactive shell if it has not already been started. It can be used to programatically start the shell from an escript or when erl is started with the -noinput or -noshell flags. The following options are allowed:

noshell | {noshell, Mode} - Starts the interactive shell as if -noshell was given to erl.

It is possible to give a Mode indicating if the input should be set in cooked or raw mode. Mode only has en effect if io:user/0 is a tty. If no Mode is given, it defaults is cooked.

When in raw mode all key presses are passed to io:user/0 as they are typed when they are typed and the characters are not echoed to the terminal. It is possible to set the echo to true using io:setopts/2 to enabling echoing again.

When in cooked mode the OS will handle the line editing and all data is passed to io:user/0 when a newline is entered.

mfa() - Starts the interactive shell using mfa() as the default shell. The mfa/0 should return the pid/0 of the created shell process.

{node(), mfa()} - Starts the interactive shell using mfa() on node() as the default shell. The mfa/0 should return the pid/0 of the created shell process.

{remote, string()} - Starts the interactive shell using as if -remsh was given to erl.

{remote, string(), mfa()} - Starts the interactive shell using as if -remsh was given to erl but with an alternative shell implementation.

On error this function will return:

already_started - if an interactive shell is already started.

noconnection - if a remote shell was requested but it could not be connected to.

badfile | nofile | on_load_failure - if a remote shell was requested with a custom mfa(), but the module could not be loaded. See Error Reasons for Code-Loading Functions for a description of the error reasons.

start_restricted(Module)

-spec start_restricted(Module) -> {error, Reason}
                          when Module :: module(), Reason :: code:load_error_rsn().

Exits a normal shell and starts a restricted shell. Module specifies the callback module for the functions local_allowed/3 and non_local_allowed/3. The function is meant to be called from the shell.

If the callback module cannot be loaded, an error tuple is returned. The Reason in the error tuple is the one returned by the code loader when trying to load the code of the callback module.

stop_restricted()

-spec stop_restricted() -> no_return().

Exits a restricted shell and starts a normal shell. The function is meant to be called from the shell.

strings(Strings)

(since OTP R16B)

-spec strings(Strings) -> Strings2 when Strings :: boolean(), Strings2 :: boolean().

Sets pretty printing of lists to Strings. The previous value of the flag is returned.

The flag can also be set by the STDLIB application variable shell_strings. Defaults to true, which means that lists of integers are printed using the string syntax, when possible. Value false means that no lists are printed using the string syntax.

whereis()

(since OTP 26.0)

-spec whereis() -> pid() | undefined.

Returns the current shell process on the node where the calling process' group_leader is located. If that node has no shell this function will return undefined.

shell_default

Customizing the Erlang environment.

The functions in this module are called when no module name is specified in a shell command.

Consider the following shell dialog:

1> lists:reverse("abc").
"cba"
2> c(foo).
{ok, foo}

In command one, module lists is called. In command two, no module name is specified. The shell searches module user_default followed by module shell_default for function c/1.

shell_default is intended for "system wide" customizations to the shell. user_default is intended for "local" or individual user customizations.

Hint

To add your own commands to the shell, create a module called user_default and add the commands you want. Then add the following line as the first line in your .erlang file in your home directory.

code:load_abs("$PATH/user_default").

$PATH is the directory where your user_default module can be found.

Summary

Functions

help()

Print the help for all shell commands.

Functions

help()

-spec help() -> true.

Print the help for all shell commands.

shell_docs

Functions used to render EEP-48 style documentation for a shell.

This module can be used to render function and type documentation to be printed in a shell. This is the module that is used to render the documentation accessed through the shell through c:h/1,2,3). Example:

1> h(maps,new,0).

  -spec new() -> Map when Map :: #{}.

Since:
  OTP 17.0

  Returns a new empty map.

  Example:

    > maps:new().
    #{}

This module formats and renders EEP-48 documentation of the format application/erlang+html. For more information about this format see Documentation Storage in EDoc's User's Guide. It can also render any other format of "text" type, although those will be rendered as is.

Summary

Types

chunk_element()

chunk_element_attr()

chunk_element_attrs()

chunk_element_block_type()

chunk_element_inline_type()

chunk_element_type()

The HTML tags allowed in application/erlang+html.

chunk_elements()

config()

The configuration of how the documentation should be rendered.

docs_v1()

The record holding EEP-48 documentation for a module. You can use code:get_doc/1 to fetch this information from a module.

Functions

normalize(Docs)

This function can be used to do whitespace normalization of application/erlang+html documentation.

render(Module, Docs)

Equivalent to render(Module, Docs, #{}).

render(Module, DocsOrFunction, ConfigOrDocs)

Render module or function documentation.

render(Module, Function, DocsOrArity, ConfigOrDocs)

Render function documentation.

render(Module, Function, Arity, Docs, Config)

Render the documentation for a function.

render_callback(Module, Docs)

Equivalent to render_callback(Module, Docs, #{}).

render_callback(Module, DocsOrCallback, ConfigOrDocs)

Render all callbacks in a module or callback documentation.

render_callback(Module, Callback, DocsOrArity, ConfigOrDocs)

Render callback documentation.

render_callback(Module, Callback, Arity, Docs, Config)

Render the documentation of a callback in a module.

render_type(Module, Docs)

Equivalent to render_type(Module, Docs, #{}).

render_type(Module, DocsOrType, ConfigOrDocs)

Render all types in a module or type documentation.

render_type(Module, Type, DocsOrArity, ConfigOrDocs)

Render type documentation.

render_type(Module, Type, Arity, Docs, Config)

Render the documentation of a type in a module.

supported_tags()

This function can be used to find out which tags are supported by application/erlang+html documentation.

validate(Module)

This function can be used to do a basic validation of the doc content of application/erlang+html format.

Types

chunk_element()

(not exported) (since OTP 23.0)

-type chunk_element() :: {chunk_element_type(), chunk_element_attrs(), chunk_elements()} | binary().

chunk_element_attr()

(since OTP 23.0)

-type chunk_element_attr() :: {atom(), unicode:chardata()}.

chunk_element_attrs()

(not exported) (since OTP 23.0)

-type chunk_element_attrs() :: [chunk_element_attr()].

chunk_element_block_type()

(not exported) (since OTP 23.0)

-type chunk_element_block_type() ::
          p | 'div' | blockquote | br | pre | ul | ol | li | dl | dt | dd | h1 | h2 | h3 | h4 | h5 | h6.

chunk_element_inline_type()

(not exported) (since OTP 23.0)

-type chunk_element_inline_type() :: a | code | em | strong | i | b.

chunk_element_type()

(not exported) (since OTP 23.0)

-type chunk_element_type() :: chunk_element_inline_type() | chunk_element_block_type().

The HTML tags allowed in application/erlang+html.

chunk_elements()

(since OTP 23.0)

-type chunk_elements() :: [chunk_element()].

config()

(not exported) (since OTP 23.2)

-type config() :: #{encoding => unicode | latin1, columns => pos_integer(), ansi => boolean()}.

The configuration of how the documentation should be rendered.

encoding - Configure the encoding that should be used by the renderer for graphical details such as bullet-points. By default shell_docs uses the value returned by io:getopts().

ansi - Configure whether ansi escape codes should be used to render graphical details such as bold and underscore. By default shell_docs will try to determine if the receiving shell supports ansi escape codes. It is possible to override the automated check by setting the kernel configuration parameter shell_docs_ansi to a boolean/0 value.

columns - Configure how wide the target documentation should be rendered. By default shell_docs used the value returned by io:columns(). It is possible to override this default by setting the stdlib configuration parameter shell_docs_columns to a pos_integer/0 value.

docs_v1()

(not exported) (since OTP 23.0)

-type docs_v1() ::
          #docs_v1{anno :: term(),
                   beam_language :: term(),
                   format :: term(),
                   module_doc :: term(),
                   metadata :: term(),
                   docs :: term()}.

The record holding EEP-48 documentation for a module. You can use code:get_doc/1 to fetch this information from a module.

Functions

normalize(Docs)

(since OTP 23.0)

-spec normalize(Docs) -> NormalizedDocs
                   when Docs :: chunk_elements(), NormalizedDocs :: chunk_elements().

This function can be used to do whitespace normalization of application/erlang+html documentation.

render(Module, Docs)

(since OTP 23.0)

-spec render(Module, Docs) -> unicode:chardata() when Module :: module(), Docs :: docs_v1().

Equivalent to render(Module, Docs, #{}).

render(Module, DocsOrFunction, ConfigOrDocs)

(since OTP 23.0)

-spec render(Module, Docs, Config) -> unicode:chardata()
                when Module :: module(), Docs :: docs_v1(), Config :: config();
            (Module, Function, Docs) -> Res
                when
                    Module :: module(),
                    Function :: atom(),
                    Docs :: docs_v1(),
                    Res :: unicode:chardata() | {error, function_missing}.

Render module or function documentation.

Renders the module documentation if called as render(Module, Docs, Config).

Equivalent to render(Module, Function, Docs, #{}) if called as render(Module, Function, Docs).

render(Module, Function, DocsOrArity, ConfigOrDocs)

(since OTP 23.0)

-spec render(Module, Function, Docs, Config) -> Res
                when
                    Module :: module(),
                    Function :: atom(),
                    Docs :: docs_v1(),
                    Config :: config(),
                    Res :: unicode:chardata() | {error, function_missing};
            (Module, Function, Arity, Docs) -> Res
                when
                    Module :: module(),
                    Function :: atom(),
                    Arity :: arity(),
                    Docs :: docs_v1(),
                    Res :: unicode:chardata() | {error, function_missing}.

Render function documentation.

Renders the function documentation if called as render(Module, Function, Docs, Config).

Equivalent to render(Module, Function, Arity, Docs, #{}) if called as render(Module, Function, Arity, Docs).

render(Module, Function, Arity, Docs, Config)

(since OTP 23.2)

-spec render(Module, Function, Arity, Docs, Config) -> Res
                when
                    Module :: module(),
                    Function :: atom(),
                    Arity :: arity(),
                    Docs :: docs_v1(),
                    Config :: config(),
                    Res :: unicode:chardata() | {error, function_missing}.

Render the documentation for a function.

render_callback(Module, Docs)

(since OTP 23.0)

-spec render_callback(Module, Docs) -> unicode:chardata() when Module :: module(), Docs :: docs_v1().

Equivalent to render_callback(Module, Docs, #{}).

render_callback(Module, DocsOrCallback, ConfigOrDocs)

(since OTP 23.0)

-spec render_callback(Module, Docs, Config) -> unicode:chardata()
                         when Module :: module(), Docs :: docs_v1(), Config :: config();
                     (Module, Callback, Docs) -> Res
                         when
                             Module :: module(),
                             Callback :: atom(),
                             Docs :: docs_v1(),
                             Res :: unicode:chardata() | {error, callback_missing}.

Render all callbacks in a module or callback documentation.

Renders a list with all callbacks if called as render_callback(Module, Docs, Config).

Equivalent to render_callback(Module, Callback, Docs, #{}) if called as render_callback(Module, Callback, Docs).

render_callback(Module, Callback, DocsOrArity, ConfigOrDocs)

(since OTP 23.0)

-spec render_callback(Module, Callback, Docs, Config) -> Res
                         when
                             Module :: module(),
                             Callback :: atom(),
                             Docs :: docs_v1(),
                             Config :: config(),
                             Res :: unicode:chardata() | {error, callback_missing};
                     (Module, Callback, Arity, Docs) -> Res
                         when
                             Module :: module(),
                             Callback :: atom(),
                             Arity :: arity(),
                             Docs :: docs_v1(),
                             Res :: unicode:chardata() | {error, callback_missing}.

Render callback documentation.

Renders the callback documentation if called as render_callback(Module, Callback, Docs, Config).

Equivalent to render_callback(Module, Callback, Arity, Docs, #{}) if called as render_callback(Module, Callback, Arity, Docs).

render_callback(Module, Callback, Arity, Docs, Config)

(since OTP 23.2)

-spec render_callback(Module, Callback, Arity, Docs, Config) -> Res
                         when
                             Module :: module(),
                             Callback :: atom(),
                             Arity :: arity(),
                             Docs :: docs_v1(),
                             Config :: config(),
                             Res :: unicode:chardata() | {error, callback_missing}.

Render the documentation of a callback in a module.

render_type(Module, Docs)

(since OTP 23.0)

-spec render_type(Module, Docs) -> unicode:chardata() when Module :: module(), Docs :: docs_v1().

Equivalent to render_type(Module, Docs, #{}).

render_type(Module, DocsOrType, ConfigOrDocs)

(since OTP 23.0)

-spec render_type(Module, Docs, Config) -> unicode:chardata()
                     when Module :: module(), Docs :: docs_v1(), Config :: config();
                 (Module, Type, Docs) -> Res
                     when
                         Module :: module(),
                         Type :: atom(),
                         Docs :: docs_v1(),
                         Res :: unicode:chardata() | {error, type_missing}.

Render all types in a module or type documentation.

Renders a list with all types if called as render_type(Module, Docs, Config).

Equivalent to render_type(Module, Type, Docs, #{}) if called as render_type(Module, Type, Docs).

render_type(Module, Type, DocsOrArity, ConfigOrDocs)

(since OTP 23.0)

-spec render_type(Module, Type, Docs, Config) -> Res
                     when
                         Module :: module(),
                         Type :: atom(),
                         Docs :: docs_v1(),
                         Config :: config(),
                         Res :: unicode:chardata() | {error, type_missing};
                 (Module, Type, Arity, Docs) -> Res
                     when
                         Module :: module(),
                         Type :: atom(),
                         Arity :: arity(),
                         Docs :: docs_v1(),
                         Res :: unicode:chardata() | {error, type_missing}.

Render type documentation.

Renders the type documentation if called as render_type(Module, Type, Docs, Config).

Equivalent to render_type(Module, Type, Arity, Docs, #{}) if called as render_type(Module, Type, Arity, Docs).

render_type(Module, Type, Arity, Docs, Config)

(since OTP 23.2)

-spec render_type(Module, Type, Arity, Docs, Config) -> Res
                     when
                         Module :: module(),
                         Type :: atom(),
                         Arity :: arity(),
                         Docs :: docs_v1(),
                         Config :: config(),
                         Res :: unicode:chardata() | {error, type_missing}.

Render the documentation of a type in a module.

supported_tags()

(since OTP 24.0)

-spec supported_tags() -> [chunk_element_type()].

This function can be used to find out which tags are supported by application/erlang+html documentation.

validate(Module)

(since OTP 23.0)

-spec validate(Module) -> ok when Module :: module() | docs_v1().

This function can be used to do a basic validation of the doc content of application/erlang+html format.

base64

Provides base64 encode and decode, see RFC 2045.

Summary

Types

base64_alphabet()

Base 64 Encoding alphabet, see RFC 4648.

base64_binary()

Base 64 encoded binary.

base64_mode()

Selector for the Base 64 Encoding alphabet used for encoding and decoding. See RFC 4648 Sections 4 and 5.

base64_string()

Base 64 encoded string.

byte_string()

Arbitrary sequences of octets.

decode_options()

Customizes the behaviour of the decode functions.

encode_options()

Customizes the behaviour of the decode functions.

Functions

decode(Base64)

Equivalent to decode(Base64, #{}).

decode(Base64, Options)

Decodes a base64 string encoded using the standard alphabet according to RFC 4648 Section 4 to plain ASCII.

decode_to_string(Base64)

Equivalent to decode(Base64), but returns a byte_string/0.

decode_to_string(Base64, Options)

Equivalent to decode(Base64, Options), but returns a byte_string/0.

encode(Data)

Equivalent to encode(Data, #{}).

encode(Data, Options)

Encodes a plain ASCII string into base64 using the alphabet indicated by the mode option. The result is 33% larger than the data.

encode_to_string(Data)

Equivalent to encode(Data), but returns a byte_string/0.

encode_to_string(Data, Options)

Equivalent to encode(Data, Options), but returns a byte_string/0.

mime_decode(Base64)

Equivalent to mime_decode_to_string(Base64, #{}).

mime_decode(Base64, Options)

Decodes a base64 "mime" string encoded using the standard alphabet according to RFC 4648 Section 4 to plain ASCII.

mime_decode_to_string(Base64)

Equivalent to mime_decode(Base64), but returns a byte_string/0.

mime_decode_to_string(Base64, Options)

Equivalent to mime_decode(Base64, Options), but returns a byte_string/0.

Types

base64_alphabet()

(not exported)

-type base64_alphabet() :: $A..$Z | $a..$z | $0..$9 | $+ | $/ | $- | $_ | $=.

Base 64 Encoding alphabet, see RFC 4648.

base64_binary()

(not exported)

-type base64_binary() :: binary().

Base 64 encoded binary.

base64_mode()

(not exported)

-type base64_mode() :: standard | urlsafe.

Selector for the Base 64 Encoding alphabet used for encoding and decoding. See RFC 4648 Sections 4 and 5.

base64_string()

(not exported)

-type base64_string() :: [base64_alphabet()].

Base 64 encoded string.

byte_string()

(not exported)

-type byte_string() :: [byte()].

Arbitrary sequences of octets.

decode_options()

(not exported)

-type decode_options() :: #{padding => boolean(), mode => base64_mode()}.

Customizes the behaviour of the decode functions.

Default value if omitted entirely or partially is #{mode => standard, padding => true}.

The mode option can be one of the following:

standard - Default. Decode the given string using the standard base64 alphabet according to RFC 4648 Section 4, that is "+" and "/" are representing bytes 62 and 63 respectively, while "-" and "_" are illegal characters.

urlsafe - Decode the given string using the alternative "URL and Filename safe" base64 alphabet according to RFC 4648 Section 5, that is "-" and "_" are representing bytes 62 and 63 respectively, while "+" and "/" are illegal characters.

The padding option can be one of the following:

true - Default. Checks the correct number of = padding characters at the end of the encoded string.

false - Accepts an encoded string with missing = padding characters at the end.

encode_options()

(not exported)

-type encode_options() :: #{padding => boolean(), mode => base64_mode()}.

Customizes the behaviour of the decode functions.

Default value if omitted entirely or partially is #{mode => standard, padding => true}.

The mode option can be one of the following:

standard - Default. Encode the given string using the standard base64 alphabet according to RFC 4648 Section 4.

urlsafe - Encode the given string using the alternative "URL and Filename safe" base64 alphabet according to RFC 4648 Section 5.

The padding option can be one of the following:

true - Default. Appends correct number of = padding characters to the encoded string.

false - Skips appending = padding characters to the encoded string.

Functions

decode(Base64)

-spec decode(Base64) -> Data when Base64 :: base64_string() | base64_binary(), Data :: binary().

Equivalent to decode(Base64, #{}).

decode(Base64, Options)

(since OTP 26.0)

-spec decode(Base64, Options) -> Data
                when
                    Base64 :: base64_string() | base64_binary(),
                    Options :: decode_options(),
                    Data :: binary().

Decodes a base64 string encoded using the standard alphabet according to RFC 4648 Section 4 to plain ASCII.

The function will strips away any whitespace characters and check for the the correct number of = padding characters at the end of the encoded string.

See decode_options/0 for details on which options can be passed.

Example:

1> base64:decode("AQIDBA==").
<<1,2,3,4>>
2> base64:decode("AQ ID BA==").
<<1,2,3,4>>
3> base64:decode("AQIDBA=").
** exception error: missing_padding
     in function  base64:decode_list/7 (base64.erl, line 734)
        *** data to decode is missing final = padding characters, if this is intended, use the `padding => false` option
4> base64:decode("AQIDBA=", #{ padding => false }).
<<1,2,3,4>>

decode_to_string(Base64)

-spec decode_to_string(Base64) -> DataString
                          when Base64 :: base64_string() | base64_binary(), DataString :: byte_string().

Equivalent to decode(Base64), but returns a byte_string/0.

decode_to_string(Base64, Options)

(since OTP 26.0)

-spec decode_to_string(Base64, Options) -> DataString
                          when
                              Base64 :: base64_string() | base64_binary(),
                              Options :: decode_options(),
                              DataString :: byte_string().

Equivalent to decode(Base64, Options), but returns a byte_string/0.

encode(Data)

-spec encode(Data) -> Base64 when Data :: byte_string() | binary(), Base64 :: base64_binary().

Equivalent to encode(Data, #{}).

encode(Data, Options)

(since OTP 26.0)

-spec encode(Data, Options) -> Base64
                when
                    Data :: byte_string() | binary(),
                    Options :: encode_options(),
                    Base64 :: base64_binary().

Encodes a plain ASCII string into base64 using the alphabet indicated by the mode option. The result is 33% larger than the data.

See encode_options/0 for details on which options can be passed.

encode_to_string(Data)

-spec encode_to_string(Data) -> Base64String
                          when Data :: byte_string() | binary(), Base64String :: base64_string().

Equivalent to encode(Data), but returns a byte_string/0.

encode_to_string(Data, Options)

(since OTP 26.0)

-spec encode_to_string(Data, Options) -> Base64String
                          when
                              Data :: byte_string() | binary(),
                              Options :: encode_options(),
                              Base64String :: base64_string().

Equivalent to encode(Data, Options), but returns a byte_string/0.

mime_decode(Base64)

-spec mime_decode(Base64) -> Data when Base64 :: base64_string() | base64_binary(), Data :: binary().

Equivalent to mime_decode_to_string(Base64, #{}).

mime_decode(Base64, Options)

(since OTP 26.0)

-spec mime_decode(Base64, Options) -> Data
                     when
                         Base64 :: base64_string() | base64_binary(),
                         Options :: decode_options(),
                         Data :: binary().

Decodes a base64 "mime" string encoded using the standard alphabet according to RFC 4648 Section 4 to plain ASCII.

The function will strips away any illegal characters. It does not check for the the correct number of = padding characters at the end of the encoded string.

See decode_options/0 for details on which options can be passed.

Example:

1> base64:mime_decode("AQIDBA==").
<<1,2,3,4>>
2> base64:mime_decode("AQIDB=A=").
<<1,2,3,4>>

mime_decode_to_string(Base64)

-spec mime_decode_to_string(Base64) -> DataString
                               when
                                   Base64 :: base64_string() | base64_binary(),
                                   DataString :: byte_string().

Equivalent to mime_decode(Base64), but returns a byte_string/0.

mime_decode_to_string(Base64, Options)

(since OTP 26.0)

-spec mime_decode_to_string(Base64, Options) -> DataString
                               when
                                   Base64 :: base64_string() | base64_binary(),
                                   Options :: decode_options(),
                                   DataString :: byte_string().

Equivalent to mime_decode(Base64, Options), but returns a byte_string/0.

erl_error behaviour

This module provides functions for pretty-printing errors and exceptions. It is used by both the shell and by proc_lib to print exceptions.

It is possible for the module raising an error to provide additional information by calling error/3 with extra error information. More details about this mechanism is described in EEP-54.

Callback Functions

The following functions are to be exported from an Error Info handler.

Summary

Types

column()

Start column number. Default is 1.

format_fun()

A fun used to format function arguments for BIF and function calls. By default the following fun will be used

format_options()

A map with formatting options.

stack_trim_fun()

A fun used to trim the end of the stacktrace. It is called with module, function, and arity from an entry from the stacktrace. The fun is to return true if the entry should be trimmed, and false otherwise. The default value is

Callbacks

format_error(Reason, StackTrace)

This callback is called when format_exception/4 or similar functionality wants to provide extra information about an error. The Module:Function called is the one specificed by the error_info map.

Functions

format_exception(Class, Reason, StackTrace)

Equivalent to format_exception/4.

format_exception(Class, Reason, StackTrace, Options)

Format the error reason and stack back-trace caught using try ... catch in the same style as the shell formats them.

Types

column()

(not exported) (since OTP 24.0)

-type column() :: pos_integer().

Start column number. Default is 1.

format_fun()

(since OTP 24.0)

-type format_fun() :: fun((term(), column()) -> iolist()).

A fun used to format function arguments for BIF and function calls. By default the following fun will be used:

fun(Term, I) -> io_lib:print(Term, I, 80, 30) end

format_options()

(since OTP 24.0)

-type format_options() ::
          #{column => column(), stack_trim_fun => stack_trim_fun(), format_fun => format_fun()}.

A map with formatting options.

stack_trim_fun()

(since OTP 24.0)

-type stack_trim_fun() :: fun((module(), atom(), arity()) -> boolean()).

A fun used to trim the end of the stacktrace. It is called with module, function, and arity from an entry from the stacktrace. The fun is to return true if the entry should be trimmed, and false otherwise. The default value is:

fun(_, _, _) -> false end

Callbacks

format_error(Reason, StackTrace)

(since OTP 24.0)

-callback format_error(Reason, StackTrace) -> ErrorDescription
                          when
                              Reason :: term(),
                              StackTrace :: erlang:stacktrace(),
                              ArgumentPosition :: pos_integer(),
                              ErrorDescription ::
                                  #{ArgumentPosition => unicode:chardata(),
                                    general => unicode:chardata(),
                                    reason => unicode:chardata()}.

This callback is called when format_exception/4 or similar functionality wants to provide extra information about an error. The Module:Function called is the one specificed by the error_info map.

The function should return a map with additional information about what have caused the exception. The possible keys of the map are:

ArgumentPosition = pos_integer() - The position of the argument that caused the error starting at 1.

general - An error that is not associated with any argument caused the error.

reason - If the Reason should be printed differently than the default way.

If the text returned includes new-lines, format_exception/4 will indent the text correctly.

Example:

-module(my_error_module).
-export([atom_to_string/1, format_error/2]).

atom_to_string(Arg) when is_atom(Arg) ->
  atom_to_list(Arg);
atom_to_string(Arg) ->
  erlang:error(badarg,[Arg],
               [{error_info,#{ module => ?MODULE,
                               cause => #{ 1 => "should be an atom" }}}]).

format_error(Reason, [{_M,_F,_As,Info}|_]) ->
  ErrorInfo = proplists:get_value(error_info, Info, #{}),
  ErrorMap = maps:get(cause, ErrorInfo),
  ErrorMap#{ general => "optional general information",
             reason => io_lib:format("~p: ~p",[?MODULE, Reason]) }.

1> c(my_error_module).
{ok,my_error_module}
2> my_error_module:atom_to_string(1).
** exception error: my_error_module: badarg
     in function  my_error_module:atom_to_string/1
        called as my_error_module:atom_to_string(1)
        *** argument 1: should be an atom
        *** optional general information

Functions

format_exception(Class, Reason, StackTrace)

(since OTP 24.0)

-spec format_exception(Class, Reason, StackTrace) -> unicode:chardata()
                          when
                              Class :: error | exit | throw,
                              Reason :: term(),
                              StackTrace :: erlang:stacktrace().

Equivalent to format_exception/4.

format_exception(Class, Reason, StackTrace, Options)

(since OTP 24.0)

-spec format_exception(Class, Reason, StackTrace, Options) -> unicode:chardata()
                          when
                              Class :: error | exit | throw,
                              Reason :: term(),
                              StackTrace :: erlang:stacktrace(),
                              Options :: format_options().

Format the error reason and stack back-trace caught using try ... catch in the same style as the shell formats them.

Example:

try
    do_something()
catch
    C:R:Stk ->
        Message = erl_error:format_exception(C, R, Stk),
        io:format(LogFile, "~ts\n", [Message])
end

If error_info is provided with the exception, format_exception will use that information to provide additional information about the exception.

Example:

try
  erlang:raise(badarg,[],[{error_info,#{}}])
catch
    C:R:Stk ->
        Message = erl_error:format_exception(C, R, Stk),
        io:format(LogFile, "~ts\n", [Message])
end

See erlang:error/3 for details on how to raise an exception with error_info included.

file_sorter

File sorter.

This module contains functions for sorting terms on files, merging already sorted files, and checking files for sortedness. Chunks containing binary terms are read from a sequence of files, sorted internally in memory and written on temporary files, which are merged producing one sorted file as output. Merging is provided as an optimization; it is faster when the files are already sorted, but it always works to sort instead of merge.

On a file, a term is represented by a header and a binary. Two options define the format of terms on files:

{header, HeaderLength} - HeaderLength determines the number of bytes preceding each binary and containing the length of the binary in bytes. Defaults to 4. The order of the header bytes is defined as follows: if B is a binary containing a header only, size Size of the binary is calculated as <<Size:HeaderLength/unit:8>> = B.

{format, Format} - Option Format determines the function that is applied to binaries to create the terms to be sorted. Defaults to binary_term, which is equivalent to fun binary_to_term/1. Value binary is equivalent to fun(X) -> X end, which means that the binaries are sorted as they are. This is the fastest format. If Format is term, io:read/2 is called to read terms. In that case, only the default value of option header is allowed.

Option format also determines what is written to the sorted output file: if Format is term, then io:format/3 is called to write each term, otherwise the binary prefixed by a header is written. Notice that the binary written is the same binary that was read; the results of applying function Format are thrown away when the terms have been sorted. Reading and writing terms using the io module is much slower than reading and writing binaries.

Other options are:

{order, Order} - The default is to sort terms in ascending order, but that can be changed by value descending or by specifying an ordering function Fun. An ordering function is antisymmetric, transitive, and total. Fun(A, B) is to return true if A comes before B in the ordering, otherwise false. An example of a typical ordering function is less than or equal to, =</2. Using an ordering function slows down the sort considerably. Functions keysort, keymerge and keycheck do not accept ordering functions.

{unique, boolean()} - When sorting or merging files, only the first of a sequence of terms that compare equal (==) is output if this option is set to true. Defaults to false, which implies that all terms that compare equal are output. When checking files for sortedness, a check that no pair of consecutive terms compares equal is done if this option is set to true.

{tmpdir, TempDirectory} - The directory where temporary files are put can be chosen explicitly. The default, implied by value "", is to put temporary files on the same directory as the sorted output file. If output is a function (see below), the directory returned by file:get_cwd() is used instead. The names of temporary files are derived from the Erlang nodename (node/0), the process identifier of the current Erlang emulator (os:getpid()), and a unique integer (erlang:unique_integer([positive])). A typical name is fs_mynode@myhost_1763_4711.17, where 17 is a sequence number. Existing files are overwritten. Temporary files are deleted unless some uncaught EXIT signal occurs.

{compressed, boolean()} - Temporary files and the output file can be compressed. Defaults false, which implies that written files are not compressed. Regardless of the value of option compressed, compressed files can always be read. Notice that reading and writing compressed files are significantly slower than reading and writing uncompressed files.

{size, Size} - By default about 512*1024 bytes read from files are sorted internally. This option is rarely needed.

{no_files, NoFiles} - By default 16 files are merged at a time. This option is rarely needed.

As an alternative to sorting files, a function of one argument can be specified as input. When called with argument read, the function is assumed to return either of the following:

end_of_input or {end_of_input, Value}} when there is no more input (Value is explained below).

{Objects, Fun}, where Objects is a list of binaries or terms depending on the format, and Fun is a new input function.

Any other value is immediately returned as value of the current call to sort or keysort. Each input function is called exactly once. If an error occurs, the last function is called with argument close, the reply of which is ignored.

A function of one argument can be specified as output. The results of sorting or merging the input is collected in a non-empty sequence of variable length lists of binaries or terms depending on the format. The output function is called with one list at a time, and is assumed to return a new output function. Any other return value is immediately returned as value of the current call to the sort or merge function. Each output function is called exactly once. When some output function has been applied to all of the results or an error occurs, the last function is called with argument close, and the reply is returned as value of the current call to the sort or merge function.

If a function is specified as input and the last input function returns {end_of_input, Value}, the function specified as output is called with argument {value, Value}. This makes it easy to initiate the sequence of output functions with a value calculated by the input functions.

As an example, consider sorting the terms on a disk log file. A function that reads chunks from the disk log and returns a list of binaries is used as input. The results are collected in a list of terms.

sort(Log) ->
    {ok, _} = disk_log:open([{name,Log}, {mode,read_only}]),
    Input = input(Log, start),
    Output = output([]),
    Reply = file_sorter:sort(Input, Output, {format,term}),
    ok = disk_log:close(Log),
    Reply.

input(Log, Cont) ->
    fun(close) ->
            ok;
       (read) ->
            case disk_log:chunk(Log, Cont) of
                {error, Reason} ->
                    {error, Reason};
                {Cont2, Terms} ->
                    {Terms, input(Log, Cont2)};
                {Cont2, Terms, _Badbytes} ->
                    {Terms, input(Log, Cont2)};
                eof ->
                    end_of_input
            end
    end.

output(L) ->
    fun(close) ->
            lists:append(lists:reverse(L));
       (Terms) ->
            output([Terms | L])
    end.

For more examples of functions as input and output, see the end of the file_sorter module; the term format is implemented with functions.

The possible values of Reason returned when an error occurs are:

bad_object, {bad_object, FileName} - Applying the format function failed for some binary, or the key(s) could not be extracted from some term.

{bad_term, FileName} - io:read/2 failed to read some term.

{file_error, FileName, file:posix()} - For an explanation of file:posix(), see file.

{premature_eof, FileName} - End-of-file was encountered inside some binary term.

Summary

Types

file_name()

file_names()

format()

format_fun()

header_length()

i_command()

i_reply()

infun()

input()

input_reply()

key_pos()

no_files()

o_command()

o_reply()

object()

option()

options()

order()

order_fun()

outfun()

output()

output_reply()

reason()

size()

tmp_directory()

value()

Functions

check(FileName)

Equivalent to check([FileName], []).

check(FileNames, Options)

Checks files for sortedness. If a file is not sorted, the first out-of-order element is returned. The first term on a file has position 1.

keycheck(KeyPos, FileName)

Equivalent to keycheck(KeyPos, [Filename], []).

keycheck(KeyPos, FileNames, Options)

Checks files for sortedness. If a file is not sorted, the first out-of-order element is returned. The first term on a file has position 1.

keymerge(KeyPos, FileNames, Output)

Equivalent to keymerge(KeyPos, FileNames, Output, []).

keymerge(KeyPos, FileNames, Output, Options)

Merges tuples on files. Each input file is assumed to be sorted on key(s).

keysort(KeyPos, FileName)

Sorts tuples on files.

keysort(KeyPos, Input, Output)

Equivalent to keysort(KeyPos, Input, Output, []).

keysort(KeyPos, Input, Output, Options)

Sorts tuples on files. The sort is performed on the element(s) mentioned in KeyPos. If two tuples compare equal (==) on one element, the next element according to KeyPos is compared. The sort is stable.

merge(FileNames, Output)

Equivalent to merge(FileNames, Output, []).

merge(FileNames, Output, Options)

Merges terms on files. Each input file is assumed to be sorted.

sort(FileName)

Sorts terms on files.

sort(Input, Output)

Equivalent to sort(Input, Output, []).

sort(Input, Output, Options)

Sorts terms on files.

Types

file_name()

(not exported)

-type file_name() :: file:name().

file_names()

(not exported)

-type file_names() :: [file:name()].

format()

(not exported)

-type format() :: binary_term | term | binary | format_fun().

format_fun()

(not exported)

-type format_fun() :: fun((binary()) -> term()).

header_length()

(not exported)

-type header_length() :: pos_integer().

i_command()

(not exported)

-type i_command() :: read | close.

i_reply()

(not exported)

-type i_reply() :: end_of_input | {end_of_input, value()} | {[object()], infun()} | input_reply().

infun()

(not exported)

-type infun() :: fun((i_command()) -> i_reply()).

input()

(not exported)

-type input() :: file_names() | infun().

input_reply()

(not exported)

-type input_reply() :: term().

key_pos()

(not exported)

-type key_pos() :: pos_integer() | [pos_integer()].

no_files()

(not exported)

-type no_files() :: pos_integer().

o_command()

(not exported)

-type o_command() :: {value, value()} | [object()] | close.

o_reply()

(not exported)

-type o_reply() :: outfun() | output_reply().

object()

(not exported)

-type object() :: term() | binary().

option()

(not exported)

-type option() ::
          {compressed, boolean()} |
          {header, header_length()} |
          {format, format()} |
          {no_files, no_files()} |
          {order, order()} |
          {size, size()} |
          {tmpdir, tmp_directory()} |
          {unique, boolean()}.

options()

(not exported)

-type options() :: [option()] | option().

order()

(not exported)

-type order() :: ascending | descending | order_fun().

order_fun()

(not exported)

-type order_fun() :: fun((term(), term()) -> boolean()).

outfun()

(not exported)

-type outfun() :: fun((o_command()) -> o_reply()).

output()

(not exported)

-type output() :: file_name() | outfun().

output_reply()

(not exported)

-type output_reply() :: term().

reason()

-type reason() ::
          bad_object |
          {bad_object, file_name()} |
          {bad_term, file_name()} |
          {file_error, file_name(), file:posix() | badarg | system_limit} |
          {premature_eof, file_name()}.

size()

(not exported)

-type size() :: non_neg_integer().

tmp_directory()

(not exported)

-type tmp_directory() :: [] | file:name().

value()

(not exported)

-type value() :: term().

Functions

check(FileName)

-spec check(FileName) -> Reply
               when
                   FileName :: file_name(),
                   Reply :: {ok, [Result]} | {error, reason()},
                   Result :: {FileName, TermPosition, term()},
                   TermPosition :: pos_integer().

Equivalent to check([FileName], []).

check(FileNames, Options)

-spec check(FileNames, Options) -> Reply
               when
                   FileNames :: file_names(),
                   Options :: options(),
                   Reply :: {ok, [Result]} | {error, reason()},
                   Result :: {FileName, TermPosition, term()},
                   FileName :: file_name(),
                   TermPosition :: pos_integer().

Checks files for sortedness. If a file is not sorted, the first out-of-order element is returned. The first term on a file has position 1.

keycheck(KeyPos, FileName)

-spec keycheck(KeyPos, FileName) -> Reply
                  when
                      KeyPos :: key_pos(),
                      FileName :: file_name(),
                      Reply :: {ok, [Result]} | {error, reason()},
                      Result :: {FileName, TermPosition, term()},
                      TermPosition :: pos_integer().

Equivalent to keycheck(KeyPos, [Filename], []).

keycheck(KeyPos, FileNames, Options)

-spec keycheck(KeyPos, FileNames, Options) -> Reply
                  when
                      KeyPos :: key_pos(),
                      FileNames :: file_names(),
                      Options :: options(),
                      Reply :: {ok, [Result]} | {error, reason()},
                      Result :: {FileName, TermPosition, term()},
                      FileName :: file_name(),
                      TermPosition :: pos_integer().

Checks files for sortedness. If a file is not sorted, the first out-of-order element is returned. The first term on a file has position 1.

keymerge(KeyPos, FileNames, Output)

-spec keymerge(KeyPos, FileNames, Output) -> Reply
                  when
                      KeyPos :: key_pos(),
                      FileNames :: file_names(),
                      Output :: output(),
                      Reply :: ok | {error, reason()} | output_reply().

Equivalent to keymerge(KeyPos, FileNames, Output, []).

keymerge(KeyPos, FileNames, Output, Options)

-spec keymerge(KeyPos, FileNames, Output, Options) -> Reply
                  when
                      KeyPos :: key_pos(),
                      FileNames :: file_names(),
                      Output :: output(),
                      Options :: options(),
                      Reply :: ok | {error, reason()} | output_reply().

Merges tuples on files. Each input file is assumed to be sorted on key(s).

keysort(KeyPos, FileName)

-spec keysort(KeyPos, FileName) -> Reply
                 when
                     KeyPos :: key_pos(),
                     FileName :: file_name(),
                     Reply :: ok | {error, reason()} | input_reply() | output_reply().

Sorts tuples on files.

keysort(KeyPos, Input, Output)

-spec keysort(KeyPos, Input, Output) -> Reply
                 when
                     KeyPos :: key_pos(),
                     Input :: input(),
                     Output :: output(),
                     Reply :: ok | {error, reason()} | input_reply() | output_reply().

Equivalent to keysort(KeyPos, Input, Output, []).

keysort(KeyPos, Input, Output, Options)

-spec keysort(KeyPos, Input, Output, Options) -> Reply
                 when
                     KeyPos :: key_pos(),
                     Input :: input(),
                     Output :: output(),
                     Options :: options(),
                     Reply :: ok | {error, reason()} | input_reply() | output_reply().

Sorts tuples on files. The sort is performed on the element(s) mentioned in KeyPos. If two tuples compare equal (==) on one element, the next element according to KeyPos is compared. The sort is stable.

merge(FileNames, Output)

-spec merge(FileNames, Output) -> Reply
               when
                   FileNames :: file_names(),
                   Output :: output(),
                   Reply :: ok | {error, reason()} | output_reply().

Equivalent to merge(FileNames, Output, []).

merge(FileNames, Output, Options)

-spec merge(FileNames, Output, Options) -> Reply
               when
                   FileNames :: file_names(),
                   Output :: output(),
                   Options :: options(),
                   Reply :: ok | {error, reason()} | output_reply().

Merges terms on files. Each input file is assumed to be sorted.

sort(FileName)

-spec sort(FileName) -> Reply
              when
                  FileName :: file_name(),
                  Reply :: ok | {error, reason()} | input_reply() | output_reply().

Sorts terms on files.

sort(Input, Output)

-spec sort(Input, Output) -> Reply
              when
                  Input :: input(),
                  Output :: output(),
                  Reply :: ok | {error, reason()} | input_reply() | output_reply().

Equivalent to sort(Input, Output, []).

sort(Input, Output, Options)

-spec sort(Input, Output, Options) -> Reply
              when
                  Input :: input(),
                  Output :: output(),
                  Options :: options(),
                  Reply :: ok | {error, reason()} | input_reply() | output_reply().

Sorts terms on files.

filelib

File utilities, such as wildcard matching of filenames.

This module contains utilities on a higher level than the file module.

This module does not support "raw" filenames (that is, files whose names do not comply with the expected encoding). Such files are ignored by the functions in this module.

For more information about raw filenames, see the file module.

Note

Functionality in this module generally assumes valid input and does not necessarily fail on input that does not use a valid encoding, but may instead very likely produce invalid output.

File operations used to accept filenames containing null characters (integer value zero). This caused the name to be truncated and in some cases arguments to primitive operations to be mixed up. Filenames containing null characters inside the filename are now rejected and will cause primitive file operations to fail.

Warning

Currently null characters at the end of the filename will be accepted by primitive file operations. Such filenames are however still documented as invalid. The implementation will also change in the future and reject such filenames.

Summary

Types

dirname()

dirname_all()

filename()

filename_all()

find_file_rule()

find_source_rule()

Functions

ensure_dir(Name)

Ensures that all parent directories for the specified file or directory name Name exist, trying to create them if necessary.

ensure_path(Path)

Ensures that all parent directories for the specified path Path exist, trying to create them if necessary.

file_size(Filename)

Returns the size of the specified file.

find_file(Filename, Dir)

Equivalent to find_file(Filename, Dir, []).

find_file/3

Looks for a file of the given name by applying suffix rules to the given directory path.

find_source(FilePath)

Equivalent to find_source(Base, Dir), where Dir is filename:dirname(FilePath) and Base is filename:basename(FilePath).

find_source(Filename, Dir)

Equivalent to find_source(Filename, Dir, []).

find_source/3

Applies file extension specific rules to find the source file for a given object file relative to the object directory.

fold_files(Dir, RegExp, Recursive, Fun, AccIn)

Folds function Fun over all (regular) files F in directory Dir whose basename (for example, just "baz.erl" in "foo/bar/baz.erl") matches the regular expression RegExp (for a description of the allowed regular expressions, see the re module).

is_dir(Name)

Returns true if Name refers to a directory, otherwise false.

is_file(Name)

Returns true if Name refers to a file or a directory, otherwise false.

is_regular(Name)

Returns true if Name refers to a (regular) file, otherwise false.

last_modified(Name)

Returns the date and time the specified file or directory was last modified, or 0 if the file does not exist.

safe_relative_path(Filename, Cwd)

Sanitizes the relative path by eliminating ".." and "." components to protect against directory traversal attacks.

wildcard(Wildcard)

Returns a list of all files that match Unix-style wildcard string Wildcard.

wildcard(Wildcard, Cwd)

Same as wildcard/1, except that Cwd is used instead of the working directory.

Types

dirname()

(not exported)

-type dirname() :: filename().

dirname_all()

(not exported)

-type dirname_all() :: filename_all().

filename()

(not exported)

-type filename() :: file:name().

filename_all()

(not exported)

-type filename_all() :: file:name_all().

find_file_rule()

(not exported)

-type find_file_rule() :: {ObjDirSuffix :: string(), SrcDirSuffix :: string()}.

find_source_rule()

(not exported)

-type find_source_rule() :: {ObjExtension :: string(), SrcExtension :: string(), [find_file_rule()]}.

Functions

ensure_dir(Name)

-spec ensure_dir(Name) -> ok | {error, Reason}
                    when Name :: filename_all() | dirname_all(), Reason :: file:posix().

Ensures that all parent directories for the specified file or directory name Name exist, trying to create them if necessary.

Returns ok if all parent directories already exist or can be created. Returns {error, Reason} if some parent directory does not exist and cannot be created.

ensure_path(Path)

(since OTP 25.0)

-spec ensure_path(Path) -> ok | {error, Reason} when Path :: dirname_all(), Reason :: file:posix().

Ensures that all parent directories for the specified path Path exist, trying to create them if necessary.

Unlike ensure_dir/1, this function will attempt to create all path segments as a directory, including the last segment.

Returns ok if all parent directories already exist or can be created. Returns {error, Reason} if some parent directory does not exist and cannot be created.

file_size(Filename)

-spec file_size(Filename) -> non_neg_integer() when Filename :: filename_all().

Returns the size of the specified file.

find_file(Filename, Dir)

(since OTP 20.0)

-spec find_file(Filename :: filename(), Dir :: filename()) -> {ok, filename()} | {error, not_found}.

Equivalent to find_file(Filename, Dir, []).

find_file/3

(since OTP 20.0)

-spec find_file(filename(), filename(), [find_file_rule()]) -> {ok, filename()} | {error, not_found}.

Looks for a file of the given name by applying suffix rules to the given directory path.

For example, a rule {"ebin", "src"} means that if the directory path ends with "ebin", the corresponding path ending in "src" should be searched.

If Rules is left out or is an empty list, the default system rules are used. See also the Kernel application parameter source_search_rules.

find_source(FilePath)

(since OTP 20.0)

-spec find_source(filename()) -> {ok, filename()} | {error, not_found}.

Equivalent to find_source(Base, Dir), where Dir is filename:dirname(FilePath) and Base is filename:basename(FilePath).

find_source(Filename, Dir)

(since OTP 20.0)

-spec find_source(filename(), filename()) -> {ok, filename()} | {error, not_found}.

Equivalent to find_source(Filename, Dir, []).

find_source/3

(since OTP 20.0)

-spec find_source(filename(), filename(), [find_source_rule()]) -> {ok, filename()} | {error, not_found}.

Applies file extension specific rules to find the source file for a given object file relative to the object directory.

For example, for a file with the extension .beam, the default rule is to look for a file with a corresponding extension .erl by replacing the suffix "ebin" of the object directory path with "src" or "src/*". The file search is done through find_file/3. The directory of the object file is always tried before any other directory specified by the rules.

If Rules is left out or is an empty list, the default system rules are used. See also the Kernel application parameter source_search_rules.

fold_files(Dir, RegExp, Recursive, Fun, AccIn)

-spec fold_files(Dir, RegExp, Recursive, Fun, AccIn) -> AccOut
                    when
                        Dir :: dirname(),
                        RegExp :: string(),
                        Recursive :: boolean(),
                        Fun :: fun((F :: file:filename(), AccIn) -> AccOut),
                        AccIn :: term(),
                        AccOut :: term().

Folds function Fun over all (regular) files F in directory Dir whose basename (for example, just "baz.erl" in "foo/bar/baz.erl") matches the regular expression RegExp (for a description of the allowed regular expressions, see the re module).

If Recursive is true, all subdirectories to Dir are processed. The regular expression matching is only done on the filename without the directory part.

If Unicode filename translation is in effect and the file system is transparent, filenames that cannot be interpreted as Unicode can be encountered, in which case the fun() must be prepared to handle raw filenames (that is, binaries). If the regular expression contains codepoints > 255, it does not match filenames that do not conform to the expected character encoding (that is, are not encoded in valid UTF-8).

For more information about raw filenames, see the file module.

is_dir(Name)

-spec is_dir(Name) -> boolean() when Name :: filename_all() | dirname_all().

Returns true if Name refers to a directory, otherwise false.

is_file(Name)

-spec is_file(Name) -> boolean() when Name :: filename_all() | dirname_all().

Returns true if Name refers to a file or a directory, otherwise false.

is_regular(Name)

-spec is_regular(Name) -> boolean() when Name :: filename_all().

Returns true if Name refers to a (regular) file, otherwise false.

last_modified(Name)

-spec last_modified(Name) -> file:date_time() | 0 when Name :: filename_all() | dirname_all().

Returns the date and time the specified file or directory was last modified, or 0 if the file does not exist.

safe_relative_path(Filename, Cwd)

(since OTP 23.0)

-spec safe_relative_path(Filename, Cwd) -> unsafe | SafeFilename
                            when
                                Filename :: filename_all(),
                                Cwd :: filename_all(),
                                SafeFilename :: filename_all().

Sanitizes the relative path by eliminating ".." and "." components to protect against directory traversal attacks.

Either returns the sanitized path name, or the atom unsafe if the path is unsafe. The path is considered unsafe in the following circumstances:

The path is not relative.

A ".." component would climb up above the root of the relative path.

A symbolic link in the path points above the root of the relative path.

Examples:

1> {ok, Cwd} = file:get_cwd().
...
2> filelib:safe_relative_path("dir/sub_dir/..", Cwd).
"dir"
3> filelib:safe_relative_path("dir/..", Cwd).
[]
4> filelib:safe_relative_path("dir/../..", Cwd).
unsafe
5> filelib:safe_relative_path("/abs/path", Cwd).
unsafe

wildcard(Wildcard)

-spec wildcard(Wildcard) -> [file:filename()] when Wildcard :: filename() | dirname().

Returns a list of all files that match Unix-style wildcard string Wildcard.

The wildcard string looks like an ordinary filename, except that the following "wildcard characters" are interpreted in a special way:

? - Matches one character.

* - Matches any number of characters up to the end of the filename, the next dot, or the next slash.

** - Two adjacent * used as a single pattern match all files and zero or more directories and subdirectories.

[Character1,Character2,...] - Matches any of the characters listed. Two characters separated by a hyphen match a range of characters. Example: [A-Z] matches any uppercase letter.

{Item,...} - Alternation. Matches one of the alternatives.

Other characters represent themselves. Only filenames that have exactly the same character in the same position match. Matching is case-sensitive, for example, "a" does not match "A".

Directory separators must always be written as /, even on Windows.

A character preceded by \ loses its special meaning. Note that \ must be written as \\ in a string literal. For example, "\\?*" will match any filename starting with ?.

Notice that multiple "*" characters are allowed (as in Unix wildcards, but opposed to Windows/DOS wildcards).

Examples:

The following examples assume that the current directory is the top of an Erlang/OTP installation.

To find all .beam files in all applications, use the following line:

filelib:wildcard("lib/*/ebin/*.beam").

To find .erl or .hrl in all applications src directories, use either of the following lines:

filelib:wildcard("lib/*/src/*.?rl")

filelib:wildcard("lib/*/src/*.{erl,hrl}")

To find all .hrl files in src or include directories:

filelib:wildcard("lib/*/{src,include}/*.hrl").

To find all .erl or .hrl files in either src or include directories:

filelib:wildcard("lib/*/{src,include}/*.{erl,hrl}")

To find all .erl or .hrl files in any subdirectory:

filelib:wildcard("lib/**/*.{erl,hrl}")

wildcard(Wildcard, Cwd)

-spec wildcard(Wildcard, Cwd) -> [file:filename()]
                  when Wildcard :: filename() | dirname(), Cwd :: dirname().

Same as wildcard/1, except that Cwd is used instead of the working directory.

filename

Filename manipulation functions.

This module provides functions for analyzing and manipulating filenames. These functions are designed so that the Erlang code can work on many different platforms with different filename formats. With filename is meant all strings that can be used to denote a file. The filename can be a short relative name like foo.erl, a long absolute name including a drive designator, a directory name like D:\usr/local\bin\erl/lib\tools\foo.erl, or any variations in between.

In Windows, all functions return filenames with forward slashes only, even if the arguments contain backslashes. To normalize a filename by removing redundant directory separators, use join/1.

The module supports raw filenames in the way that if a binary is present, or the filename cannot be interpreted according to the return value of file:native_name_encoding/0, a raw filename is also returned. For example, join/1 provided with a path component that is a binary (and cannot be interpreted under the current native filename encoding) results in a raw filename that is returned (the join operation is performed of course). For more information about raw filenames, see the file module.

Note

Functionality in this module generally assumes valid input and does not necessarily fail on input that does not use a valid encoding, but may instead very likely produce invalid output.

File operations used to accept filenames containing null characters (integer value zero). This caused the name to be truncated and in some cases arguments to primitive operations to be mixed up. Filenames containing null characters inside the filename are now rejected and will cause primitive file operations to fail.

Warning

Currently null characters at the end of the filename will be accepted by primitive file operations. Such filenames are however still documented as invalid. The implementation will also change in the future and reject such filenames.

Summary

Types

basedir_opts()

basedir_path_type()

basedir_paths_type()

Functions

absname(Filename)

Converts a relative Filename and returns an absolute name. No attempt is made to create the shortest absolute name, as this can give incorrect results on file systems that allow links.

absname(Filename, Dir)

Same as absname/1, except that the directory to which the filename is to be made relative is specified in argument Dir.

absname_join(Dir, Filename)

Joins an absolute directory with a relative filename.

basedir(Type, Application)

Equivalent to basedir(PathType, Application, #{}) or basedir(PathsType, Application, #{}).

basedir(Type, Application, Opts)

Returns a suitable path, or paths, for a given type.

basename(Filename)

Returns the last component of Filename, or Filename itself if it does not contain any directory separators.

basename(Filename, Ext)

Returns the last component of Filename with extension Ext stripped.

dirname(Filename)

Returns the directory part of Filename.

extension(Filename)

Returns the file extension of Filename, including the period. Returns an empty string if no extension exists.

flatten(Filename)

Converts a possibly deep list filename consisting of characters and atoms into the corresponding flat string filename.

join(Components)

Joins a list of filename Components with directory separators. If one of the elements of Components includes an absolute path, such as "/xxx", the preceding elements, if any, are removed from the result.

join(Name1, Name2)

Joins two filename components with directory separators. Equivalent to join([Name1, Name2]).

nativename(Path)

Converts Path to a form accepted by the command shell and native applications on the current platform. On Windows, forward slashes are converted to backward slashes. On all platforms, the name is normalized as done by join/1.

pathtype(Path)

Returns the path type, which is one of the following

rootname(Filename)

Removes the filename extension.

rootname(Filename, Ext)

Removes the filename extension Ext from Filename.

split(Filename)

Returns a list whose elements are the path components of Filename.

Types

basedir_opts()

(not exported)

-type basedir_opts() ::
          #{author => string() | binary(),
            os => windows | darwin | linux,
            version => string() | binary()}.

basedir_path_type()

(not exported)

-type basedir_path_type() :: user_cache | user_config | user_data | user_log.

basedir_paths_type()

(not exported)

-type basedir_paths_type() :: site_config | site_data.

Functions

absname(Filename)

-spec absname(Filename) -> file:filename_all() when Filename :: file:name_all().

Converts a relative Filename and returns an absolute name. No attempt is made to create the shortest absolute name, as this can give incorrect results on file systems that allow links.

Unix examples:

1> pwd().
"/usr/local"
2> filename:absname("foo").
"/usr/local/foo"
3> filename:absname("../x").
"/usr/local/../x"
4> filename:absname("/").
"/"

Windows examples:

1> pwd().
"D:/usr/local"
2> filename:absname("foo").
"D:/usr/local/foo"
3> filename:absname("../x").
"D:/usr/local/../x"
4> filename:absname("/").
"D:/"

absname(Filename, Dir)

-spec absname(Filename, Dir) -> file:filename_all()
                 when Filename :: file:name_all(), Dir :: file:name_all().

Same as absname/1, except that the directory to which the filename is to be made relative is specified in argument Dir.

absname_join(Dir, Filename)

-spec absname_join(Dir, Filename) -> file:filename_all()
                      when Dir :: file:name_all(), Filename :: file:name_all().

Joins an absolute directory with a relative filename.

Similar to join/2, but on platforms with tight restrictions on raw filename length and no support for symbolic links, leading parent directory components in Filename are matched against trailing directory components in Dir so they can be removed from the result - minimizing its length.

basedir(Type, Application)

(since OTP 19.0)

-spec basedir(PathType, Application) -> file:filename_all()
                 when PathType :: basedir_path_type(), Application :: string() | binary();
             (PathsType, Application) -> [file:filename_all()]
                 when PathsType :: basedir_paths_type(), Application :: string() | binary().

Equivalent to basedir(PathType, Application, #{}) or basedir(PathsType, Application, #{}).

basedir(Type, Application, Opts)

(since OTP 19.0)

-spec basedir(PathType, Application, Opts) -> file:filename_all()
                 when
                     PathType :: basedir_path_type(),
                     Application :: string() | binary(),
                     Opts :: basedir_opts();
             (PathsType, Application, Opts) -> [file:filename_all()]
                 when
                     PathsType :: basedir_paths_type(),
                     Application :: string() | binary(),
                     Opts :: basedir_opts().

Returns a suitable path, or paths, for a given type.

If os is not set in Opts the function will default to the native option, that is 'linux', 'darwin' or 'windows', as understood by os:type/0. Anything not recognized as 'darwin' or 'windows' is interpreted as 'linux'.

The options 'author' and 'version' are only used with 'windows' option mode.

user_cache

The path location is intended for transient data files on a local machine.

On Linux: Respects the os environment variable XDG_CACHE_HOME.

1> filename:basedir(user_cache, "my_application", #{os=>linux}).
"/home/otptest/.cache/my_application"

On Darwin:

1> filename:basedir(user_cache, "my_application", #{os=>darwin}).
"/home/otptest/Library/Caches/my_application"

On Windows:

1> filename:basedir(user_cache, "My App").
"c:/Users/otptest/AppData/Local/My App/Cache"
2> filename:basedir(user_cache, "My App").
"c:/Users/otptest/AppData/Local/My App/Cache"
3> filename:basedir(user_cache, "My App", #{author=>"Erlang"}).
"c:/Users/otptest/AppData/Local/Erlang/My App/Cache"
4> filename:basedir(user_cache, "My App", #{version=>"1.2"}).
"c:/Users/otptest/AppData/Local/My App/1.2/Cache"
5> filename:basedir(user_cache, "My App", #{author=>"Erlang",version=>"1.2"}).
"c:/Users/otptest/AppData/Local/Erlang/My App/1.2/Cache"

user_config

The path location is intended for persistent configuration files.

On Linux: Respects the os environment variable XDG_CONFIG_HOME.

2> filename:basedir(user_config, "my_application", #{os=>linux}).
"/home/otptest/.config/my_application"

On Darwin:

2> filename:basedir(user_config, "my_application", #{os=>darwin}).
"/home/otptest/Library/Application Support/my_application"

On Windows:

1> filename:basedir(user_config, "My App").
"c:/Users/otptest/AppData/Roaming/My App"
2> filename:basedir(user_config, "My App", #{author=>"Erlang", version=>"1.2"}).
"c:/Users/otptest/AppData/Roaming/Erlang/My App/1.2"

user_data

The path location is intended for persistent data files.

On Linux: Respects the os environment variable XDG_DATA_HOME.

3> filename:basedir(user_data, "my_application", #{os=>linux}).
"/home/otptest/.local/my_application"

On Darwin:

3> filename:basedir(user_data, "my_application", #{os=>darwin}).
"/home/otptest/Library/Application Support/my_application"

On Windows:

8> filename:basedir(user_data, "My App").
"c:/Users/otptest/AppData/Local/My App"
9> filename:basedir(user_data, "My App",#{author=>"Erlang",version=>"1.2"}).
"c:/Users/otptest/AppData/Local/Erlang/My App/1.2"

user_log

The path location is intended for transient log files on a local machine.

On Linux: Respects the os environment variable XDG_CACHE_HOME.

4> filename:basedir(user_log, "my_application", #{os=>linux}).
"/home/otptest/.cache/my_application/log"

On Darwin:

4> filename:basedir(user_log, "my_application", #{os=>darwin}).
"/home/otptest/Library/Logs/my_application"

On Windows:

12> filename:basedir(user_log, "My App").
"c:/Users/otptest/AppData/Local/My App/Logs"
13> filename:basedir(user_log, "My App",#{author=>"Erlang",version=>"1.2"}).
"c:/Users/otptest/AppData/Local/Erlang/My App/1.2/Logs"

site_config

On Linux: Respects the os environment variable XDG_CONFIG_DIRS.

5> filename:basedir(site_config, "my_application", #{os=>linux}).
["/usr/local/share/my_application",
 "/usr/share/my_application"]
6> os:getenv("XDG_CONFIG_DIRS").
"/etc/xdg/xdg-ubuntu:/usr/share/upstart/xdg:/etc/xdg"
7> filename:basedir(site_config, "my_application", #{os=>linux}).
["/etc/xdg/xdg-ubuntu/my_application",
 "/usr/share/upstart/xdg/my_application",
 "/etc/xdg/my_application"]
8> os:unsetenv("XDG_CONFIG_DIRS").
true
9> filename:basedir(site_config, "my_application", #{os=>linux}).
["/etc/xdg/my_application"]

On Darwin:

5> filename:basedir(site_config, "my_application", #{os=>darwin}).
["/Library/Application Support/my_application"]

site_data

On Linux: Respects the os environment variable XDG_DATA_DIRS.

10> os:getenv("XDG_DATA_DIRS").
"/usr/share/ubuntu:/usr/share/gnome:/usr/local/share/:/usr/share/"
11> filename:basedir(site_data, "my_application", #{os=>linux}).
["/usr/share/ubuntu/my_application",
 "/usr/share/gnome/my_application",
 "/usr/local/share/my_application",
 "/usr/share/my_application"]
12> os:unsetenv("XDG_DATA_DIRS").
true
13> filename:basedir(site_data, "my_application", #{os=>linux}).
["/usr/local/share/my_application",
 "/usr/share/my_application"]

On Darwin:

5> filename:basedir(site_data, "my_application", #{os=>darwin}).
["/Library/Application Support/my_application"]

basename(Filename)

-spec basename(Filename) -> file:filename_all() when Filename :: file:name_all().

Returns the last component of Filename, or Filename itself if it does not contain any directory separators.

Examples:

5> filename:basename("foo").
"foo"
6> filename:basename("/usr/foo").
"foo"
7> filename:basename("/").
[]

basename(Filename, Ext)

-spec basename(Filename, Ext) -> file:filename_all()
                  when Filename :: file:name_all(), Ext :: file:name_all().

Returns the last component of Filename with extension Ext stripped.

This function is to be used to remove a (possible) specific extension. To remove an existing extension when you are unsure which one it is, use rootname(basename(Filename)).

Examples:

8> filename:basename("~/src/kalle.erl", ".erl").
"kalle"
9> filename:basename("~/src/kalle.beam", ".erl").
"kalle.beam"
10> filename:basename("~/src/kalle.old.erl", ".erl").
"kalle.old"
11> filename:rootname(filename:basename("~/src/kalle.erl")).
"kalle"
12> filename:rootname(filename:basename("~/src/kalle.beam")).
"kalle"

dirname(Filename)

-spec dirname(Filename) -> file:filename_all() when Filename :: file:name_all().

Returns the directory part of Filename.

Examples:

13> filename:dirname("/usr/src/kalle.erl").
"/usr/src"
14> filename:dirname("kalle.erl").
"."

5> filename:dirname("\\usr\\src/kalle.erl"). % Windows
"/usr/src"

extension(Filename)

-spec extension(Filename) -> file:filename_all() when Filename :: file:name_all().

Returns the file extension of Filename, including the period. Returns an empty string if no extension exists.

Examples:

15> filename:extension("foo.erl").
".erl"
16> filename:extension("beam.src/kalle").
[]

flatten(Filename)

-spec flatten(Filename) -> file:filename_all() when Filename :: file:name_all().

Converts a possibly deep list filename consisting of characters and atoms into the corresponding flat string filename.

join(Components)

-spec join(Components) -> file:filename_all() when Components :: [file:name_all()].

Joins a list of filename Components with directory separators. If one of the elements of Components includes an absolute path, such as "/xxx", the preceding elements, if any, are removed from the result.

The result is "normalized":

Redundant directory separators are removed.

In Windows, all directory separators are forward slashes and the drive letter is in lower case.

Examples:

17> filename:join(["/usr", "local", "bin"]).
"/usr/local/bin"
18> filename:join(["a/b///c/"]).
"a/b/c"

6> filename:join(["B:a\\b///c/"]). % Windows
"b:a/b/c"

join(Name1, Name2)

-spec join(Name1, Name2) -> file:filename_all() when Name1 :: file:name_all(), Name2 :: file:name_all().

Joins two filename components with directory separators. Equivalent to join([Name1, Name2]).

nativename(Path)

-spec nativename(Path) -> file:filename_all() when Path :: file:name_all().

Converts Path to a form accepted by the command shell and native applications on the current platform. On Windows, forward slashes are converted to backward slashes. On all platforms, the name is normalized as done by join/1.

Examples:

19> filename:nativename("/usr/local/bin/"). % Unix
"/usr/local/bin"

7> filename:nativename("/usr/local/bin/"). % Windows
"\\usr\\local\\bin"

pathtype(Path)

-spec pathtype(Path) -> absolute | relative | volumerelative when Path :: file:name_all().

Returns the path type, which is one of the following:

absolute - The path name refers to a specific file on a specific volume.

Unix example: /usr/local/bin

Windows example: D:/usr/local/bin

relative - The path name is relative to the current working directory on the current volume.

Example: foo/bar, ../src

volumerelative - The path name is relative to the current working directory on a specified volume, or it is a specific file on the current working volume.

Windows example: D:bar.erl, /bar/foo.erl

rootname(Filename)

-spec rootname(Filename) -> file:filename_all() when Filename :: file:name_all().

Removes the filename extension.

Examples:

1> filename:rootname("/beam.src/kalle").
"/beam.src/kalle"
2> filename:rootname("/beam.src/foo.erl").
"/beam.src/foo"

rootname(Filename, Ext)

-spec rootname(Filename, Ext) -> file:filename_all()
                  when Filename :: file:name_all(), Ext :: file:name_all().

Removes the filename extension Ext from Filename.

Examples:

1> filename:rootname("/beam.src/foo.erl", ".erl").
"/beam.src/foo"
2> filename:rootname("/beam.src/foo.beam", ".erl").
"/beam.src/foo.beam"

split(Filename)

-spec split(Filename) -> Components when Filename :: file:name_all(), Components :: [file:name_all()].

Returns a list whose elements are the path components of Filename.

Examples:

24> filename:split("/usr/local/bin").
["/","usr","local","bin"]
25> filename:split("foo/bar").
["foo","bar"]
26> filename:split("a:\\msdev\\include").
["a:/","msdev","include"]

io

Standard I/O server interface functions.

This module provides an interface to standard Erlang I/O servers. The output functions all return ok if they are successful, or exit if they are not.

All functions in this module have an optional parameter IoDevice. If included, it must be the pid of a process that handles the I/O protocols. Normally, it is an IoDevice returned by file:open/2. If no IoDevice is given, standard_io is used.

For a description of the I/O protocols, see section The Erlang I/O Protocol in the User's Guide.

Warning

The data supplied to function put_chars/2 is to be in the unicode:chardata/0 format. This means that programs supplying binaries to this function must convert them to UTF-8 before trying to output the data on an I/O device.

If an I/O device is set in binary mode, functions get_chars/2,3 and get_line/1,2 can return binaries instead of lists. The binaries are encoded in UTF-8.

To work with binaries in ISO Latin-1 encoding, use the file module instead.

For conversion functions between character encodings, see the unicode module.

Error Information

The ErrorInfo mentioned in this module is the standard ErrorInfo structure that is returned from all I/O modules. It has the following format:

{ErrorLocation, Module, ErrorDescriptor}

A string that describes the error is obtained with the following call:

Module:format_error(ErrorDescriptor)

Summary

Types

device()

An I/O device, either standard_io/0, standard_error/0, user/0, a file:io_server/0, a registered name, or any pid handling I/O protocols.

encoding()

expand_fun()

format()

getopt()

option()

parse_form_ret()

parse_ret()

prompt()

server_no_data()

What the I/O server sends when there is no data.

setopt()

standard_error()

The I/O device standard_error can be used to direct output to whatever the current operating system considers a suitable I/O device for error output. This can be useful when standard output is redirected.

standard_io()

The default standard I/O device assigned to a process. This device is used when no IoDevice argument is specified in the function calls in this module.

user()

An I/O device that can be used to interact with the node local stdout and stdin. This can be either a terminal, a pipe, a file, or a combination.

Functions

columns()

Equivalent to columns(standard_io).

columns(IoDevice)

Retrieves the number of columns of the IoDevice (that is, the width of a terminal).

format(Format)

Equivalent to format(Format, []).

format(Format, Data)

Equivalent to format(standard_io, Format, Data).

format(IoDevice, Format, Data)

Equivalent to fwrite(IoDevice, Format, Data).

fread(Prompt, Format)

Equivalent to fread(standard_io, Prompt, Format).

fread(IoDevice, Prompt, Format)

Reads characters from IoDevice, prompting it with Prompt. Interprets the characters in accordance with Format.

fwrite(Format)

Equivalent to fwrite(Format, []).

fwrite(Format, Data)

Equivalent to fwrite(standard_io, Format, Data).

fwrite(IoDevice, Format, Data)

Writes the items in Data on the IoDevice in accordance with Format.

get_chars(Prompt, Count)

Equivalent to get_chars(standard_io, Prompt, Count).

get_chars(IoDevice, Prompt, Count)

Reads Count characters from IoDevice, prompting it with Prompt.

get_line(Prompt)

Equivalent to get_line(standard_io, Prompt).

get_line(IoDevice, Prompt)

Reads a line from IoDevice, prompting it with Prompt.

get_password()

Reads a password from user/0. Works just as get_line/2 except that the typed characters are not printed to the terminal.

getopts()

Equivalent to getopts(standard_io).

getopts(IoDevice)

Requests all available options and their current values for a IoDevice.

nl()

Equivalent to nl(standard_io).

nl(IoDevice)

Writes new line to the standard output (IoDevice).

parse_erl_exprs(Prompt)

Equivalent to parse_erl_exprs(standard_io, Prompt).

parse_erl_exprs(IoDevice, Prompt)

Equivalent to parse_erl_exprs(IoDevice, Prompt, 1).

parse_erl_exprs(IoDevice, Prompt, StartLocation)

Equivalent to parse_erl_exprs(IoDevice, Prompt, StartLocation, []).

parse_erl_exprs(IoDevice, Prompt, StartLocation, Options)

Reads data from IoDevice, prompting it with Prompt.

parse_erl_form(Prompt)

Equivalent to parse_erl_form(standard_io, Prompt).

parse_erl_form(IoDevice, Prompt)

Equivalent to parse_erl_form(IoDevice, Prompt, 1).

parse_erl_form(IoDevice, Prompt, StartLocation)

Equivalent to parse_erl_form(IoDevice, Prompt, StartLocation, []).

parse_erl_form(IoDevice, Prompt, StartLocation, Options)

Reads data from IoDevice, prompting it with Prompt.

printable_range()

Returns the user-requested range of printable Unicode characters.

put_chars(CharData)

Equivalent to put_chars(standard_io, CharData).

put_chars(IoDevice, CharData)

Writes the characters of CharData to the IoDevice.

read(Prompt)

Equivalent to read(standard_io, Prompt).

read(IoDevice, Prompt)

Reads a term Term from the standard input (IoDevice), prompting it with Prompt.

read(IoDevice, Prompt, StartLocation)

Equivalent to read(IoDevice, Prompt, StartLocation, []).

read(IoDevice, Prompt, StartLocation, Options)

Reads a term Term from IoDevice, prompting it with Prompt.

rows()

Equivalent to rows(standard_io).

rows(IoDevice)

Retrieves the number of rows of IoDevice (that is, the height of a terminal).

scan_erl_exprs(Prompt)

Equivalent to scan_erl_exprs(standard_io, Prompt).

scan_erl_exprs(Device, Prompt)

Equivalent to scan_erl_exprs(Device, Prompt, 1).

scan_erl_exprs(Device, Prompt, StartLocation)

Equivalent to scan_erl_exprs(Device, Prompt, StartLocation, []).

scan_erl_exprs(Device, Prompt, StartLocation, Options)

Reads data from IoDevice, prompting it with Prompt.

scan_erl_form(Prompt)

Equivalent to scan_erl_form(standard_io, Prompt).

scan_erl_form(IoDevice, Prompt)

Equivalent to scan_erl_form(IoDevice, Prompt, 1).

scan_erl_form(IoDevice, Prompt, StartLocation)

Equivalent to scan_erl_form(IoDevice, Prompt, StartLocation, []).

scan_erl_form(IoDevice, Prompt, StartLocation, Options)

Reads data from IoDevice, prompting it with Prompt.

setopts(Opts)

Equivalent to setopts(standard_io, Opts).

setopts(IoDevice, Opts)

Set options for IoDevice. Possible options and values vary depending on the I/O device.

write(Term)

Equivalent to write(standard_io, Term).

write(IoDevice, Term)

Writes term Term to IoDevice.

Types

device()

-type device() :: atom() | pid() | file:io_server() | standard_io() | standard_error() | user().

An I/O device, either standard_io/0, standard_error/0, user/0, a file:io_server/0, a registered name, or any pid handling I/O protocols.

encoding()

(not exported)

-type encoding() ::
          latin1 | unicode | utf8 | utf16 | utf32 | {utf16, big | little} | {utf32, big | little}.

expand_fun()

(not exported)

-type expand_fun() :: fun((string()) -> {yes | no, string(), list()}).

format()

-type format() :: atom() | string() | binary().

getopt()

(not exported)

-type getopt() :: {terminal | stdin | stdout | stderr, boolean()} | option().

option()

(not exported)

-type option() ::
          {binary, boolean()} |
          {echo, boolean()} |
          {expand_fun, expand_fun()} |
          {encoding, encoding()} |
          {atom(), term()}.

parse_form_ret()

(not exported)

-type parse_form_ret() ::
          {ok, AbsForm :: erl_parse:abstract_form(), EndLocation :: erl_anno:location()} |
          {eof, EndLocation :: erl_anno:location()} |
          {error,
           ErrorInfo :: erl_scan:error_info() | erl_parse:error_info(),
           ErrorLocation :: erl_anno:location()} |
          server_no_data().

parse_ret()

(not exported)

-type parse_ret() ::
          {ok, ExprList :: [erl_parse:abstract_expr()], EndLocation :: erl_anno:location()} |
          {eof, EndLocation :: erl_anno:location()} |
          {error,
           ErrorInfo :: erl_scan:error_info() | erl_parse:error_info(),
           ErrorLocation :: erl_anno:location()} |
          server_no_data().

prompt()

(not exported)

-type prompt() :: atom() | unicode:chardata().

server_no_data()

-type server_no_data() :: {error, ErrorDescription :: term()} | eof.

What the I/O server sends when there is no data.

setopt()

(not exported)

-type setopt() :: binary | list | option().

standard_error()

-type standard_error() :: standard_error.

The I/O device standard_error can be used to direct output to whatever the current operating system considers a suitable I/O device for error output. This can be useful when standard output is redirected.

Example on a Unix-like operating system:

$ erl -noinput -eval 'io:format(standard_error,"Error: ~s~n",["error 11"]),'\
'init:stop().' > /dev/null
Error: error 11

standard_io()

-type standard_io() :: standard_io.

The default standard I/O device assigned to a process. This device is used when no IoDevice argument is specified in the function calls in this module.

It is sometimes desirable to use an explicit IoDevice argument that refers to the default I/O device. This is the case with functions that can access either a file or the default I/O device. The atom standard_io has this special meaning. The following example illustrates this:

27> io:read('enter>').
enter>foo.
{ok,foo}
28> io:read(standard_io, 'enter>').
enter>bar.
{ok,bar}

By default all I/O sent to standard_io will end up in the user I/O device of the node that spawned the calling process.

standard_io is an alias for group_leader/0, so in order to change where the default input/output requests are sent you can change the group leader of the current process using group_leader(NewGroupLeader, self()).

user()

-type user() :: user.

An I/O device that can be used to interact with the node local stdout and stdin. This can be either a terminal, a pipe, a file, or a combination.

Use getopts/1 to get more information about the I/O device.

See The Interactive Shell and Escripts and non-interactive I/O in the Using Unicode In Erlang User's Guide for details on how Unicode is handled by user.

Functions

columns()

-spec columns() -> {ok, pos_integer()} | {error, enotsup}.

Equivalent to columns(standard_io).

columns(IoDevice)

-spec columns(IoDevice) -> {ok, pos_integer()} | {error, enotsup} when IoDevice :: device().

Retrieves the number of columns of the IoDevice (that is, the width of a terminal).

The function succeeds for terminal devices and returns {error, enotsup} for all other I/O devices.

format(Format)

-spec format(Format) -> ok when Format :: format().

Equivalent to format(Format, []).

format(Format, Data)

-spec format(Format, Data) -> ok when Format :: format(), Data :: [term()].

Equivalent to format(standard_io, Format, Data).

format(IoDevice, Format, Data)

-spec format(IoDevice, Format, Data) -> ok
                when IoDevice :: device(), Format :: format(), Data :: [term()].

Equivalent to fwrite(IoDevice, Format, Data).

fread(Prompt, Format)

-spec fread(Prompt, Format) -> Result
               when
                   Prompt :: prompt(),
                   Format :: format(),
                   Result :: {ok, Terms :: [term()]} | eof | {error, What :: term()}.

Equivalent to fread(standard_io, Prompt, Format).

fread(IoDevice, Prompt, Format)

-spec fread(IoDevice, Prompt, Format) -> Result
               when
                   IoDevice :: device(),
                   Prompt :: prompt(),
                   Format :: format(),
                   Result ::
                       {ok, Terms :: [term()]} |
                       {error, {fread, FreadError :: io_lib:fread_error()}} |
                       server_no_data().

Reads characters from IoDevice, prompting it with Prompt. Interprets the characters in accordance with Format.

Format can contain the following:

Whitespace characters (Space, Tab, and Newline) that cause input to be read to the next non-whitespace character.

Ordinary characters that must match the next input character.

Control sequences, which have the general format ~*FMC, where:

Character * is an optional return suppression character. It provides a method to specify a field that is to be omitted.

F is the field width of the input field.

M is an optional translation modifier (of which t is the only supported, meaning Unicode translation).

C determines the type of control sequence.

Unless otherwise specified, leading whitespace is ignored for all control sequences. An input field cannot be more than one line wide.Available control sequences:

~ - A single ~ is expected in the input.

d - A decimal integer is expected.

u - An unsigned integer in base 2-36 is expected. The field width parameter is used to specify base. Leading whitespace characters are not skipped.

- - An optional sign character is expected. A sign character - gives return value -1. Sign character + or none gives 1. The field width parameter is ignored. Leading whitespace characters are not skipped.

# - An integer in base 2-36 with Erlang-style base prefix (for example, "16#ffff") is expected.

f - A floating point number is expected. It must follow the Erlang floating point number syntax.

s - A string of non-whitespace characters is read. If a field width has been specified, this number of characters are read and all trailing whitespace characters are stripped. An Erlang string (list of characters) is returned.

If Unicode translation is in effect (~ts), characters > 255 are accepted, otherwise not. With the translation modifier, the returned list can as a consequence also contain integers > 255:

1> io:fread("Prompt> ","~s").
Prompt> <Characters beyond latin1 range not printable in this medium>
{error,{fread,string}}
2> io:fread("Prompt> ","~ts").
Prompt> <Characters beyond latin1 range not printable in this medium>
{ok,[[1091,1085,1080,1094,1086,1076,1077]]}

a - Similar to s, but the resulting string is converted into an atom.

c - The number of characters equal to the field width are read (default is 1) and returned as an Erlang string. However, leading and trailing whitespace characters are not omitted as they are with s. All characters are returned.

The Unicode translation modifier works as with s:

1> io:fread("Prompt> ","~c").
Prompt> <Character beyond latin1 range not printable in this medium>
{error,{fread,string}}
2> io:fread("Prompt> ","~tc").
Prompt> <Character beyond latin1 range not printable in this medium>
{ok,[[1091]]}

l - Returns the number of characters that have been scanned up to that point, including whitespace characters.

The function returns:

{ok, Terms} - The read was successful and Terms is the list of successfully matched and read items.

eof - End of file was encountered.

{error, FreadError} - The reading failed and FreadError gives a hint about the error.

{error, ErrorDescription} - The read operation failed and parameter ErrorDescription gives a hint about the error.

Examples:

20> io:fread('enter>', "~f~f~f").
enter>1.9 35.5e3 15.0
{ok,[1.9,3.55e4,15.0]}
21> io:fread('enter>', "~10f~d").
enter>     5.67899
{ok,[5.678,99]}
22> io:fread('enter>', ":~10s:~10c:").
enter>:   alan   :   joe    :
{ok, ["alan", "   joe    "]}

fwrite(Format)

-spec fwrite(Format) -> ok when Format :: format().

Equivalent to fwrite(Format, []).

fwrite(Format, Data)

-spec fwrite(Format, Data) -> ok when Format :: format(), Data :: [term()].

Equivalent to fwrite(standard_io, Format, Data).

fwrite(IoDevice, Format, Data)

-spec fwrite(IoDevice, Format, Data) -> ok
                when IoDevice :: device(), Format :: format(), Data :: [term()].

Writes the items in Data on the IoDevice in accordance with Format.

Format contains plain characters that are copied to the output device, and control sequences for formatting, see below. If Format is an atom or a binary, it is first converted to a list with the aid of atom_to_list/1 or binary_to_list/1. Example:

1> io:fwrite("Hello world!~n", []).
Hello world!
ok

The general format of a control sequence is ~F.P.PadModC.

The character C determines the type of control sequence to be used. It is the only required field. All of F, P, Pad, and Mod are optional. For example, to use a # for Pad but use the default values for F and P, you can write ~..#C.

F is the field width of the printed argument. A negative value means that the argument is left-justified within the field, otherwise right-justified. If no field width is specified, the required print width is used. If the field width specified is too small, the whole field is filled with * characters.

P is the precision of the printed argument. A default value is used if no precision is specified. The interpretation of precision depends on the control sequences. Unless otherwise specified, argument within is used to determine print width.

Pad is the padding character. This is the character used to pad the printed representation of the argument so that it conforms to the specified field width and precision. Only one padding character can be specified and, whenever applicable, it is used for both the field width and precision. The default padding character is ' ' (space).

Mod is the control sequence modifier. This is one or more characters that change the interpretation of Data.

The current modifiers are:

t - For Unicode translation.

l - For stopping p and P from detecting printable characters.

k - For use with p, P, w, and W to format maps in map-key ordered order (see maps:iterator_order/0).

K - Similar to k, for formatting maps in map-key order, but takes an extra argument that specifies the maps:iterator_order/0.

For example:

> M = #{ a => 1, b => 2 }.
#{a => 1,b => 2}
> io:format("~Kp~n", [reversed, M]).
#{b => 2,a => 1}
ok

If F, P, or Pad is a * character, the next argument in Data is used as the value. For example:

1> io:fwrite("~*.*.0f~n",[9, 5, 3.14159265]).
003.14159
ok

To use a literal * character as Pad, it must be passed as an argument:

2> io:fwrite("~*.*.*f~n",[9, 5, $*, 3.14159265]).
**3.14159
ok

Available control sequences:

~ - Character ~ is written.

c - The argument is a number that is interpreted as an ASCII code. The precision is the number of times the character is printed and defaults to the field width, which in turn defaults to 1. Example:

1> io:fwrite("|~10.5c|~-10.5c|~5c|~n", [$a, $b, $c]).
|     aaaaa|bbbbb     |ccccc|
ok

If the Unicode translation modifier (t) is in effect, the integer argument can be any number representing a valid Unicode codepoint, otherwise it is to be an integer less than or equal to 255, otherwise it is masked with 16#FF:

2> io:fwrite("~tc~n",[1024]).
\x{400}
ok
3> io:fwrite("~c~n",[1024]).
^@
ok

f - The argument is a float that is written as [-]ddd.ddd, where the precision is the number of digits after the decimal point. The default precision is 6 and it cannot be < 1.

e - The argument is a float that is written as [-]d.ddde+-ddd, where the precision is the number of digits written. The default precision is 6 and it cannot be < 2.

g - The argument is a float that is written as f, if it is >= 0.1 and < 10000.0. Otherwise, it is written in the e format. The precision is the number of significant digits. It defaults to 6 and is not to be < 2. If the absolute value of the float does not allow it to be written in the f format with the desired number of significant digits, it is also written in the e format.

s - Prints the argument with the string syntax. The argument is, if no Unicode translation modifier is present, an iolist/0, a binary/0, or an atom/0. If the Unicode translation modifier (t) is in effect, the argument is unicode:chardata(), meaning that binaries are in UTF-8. The characters are printed without quotes. The string is first truncated by the specified precision and then padded and justified to the specified field width. The default precision is the field width.

This format can be used for printing any object and truncating the output so it fits a specified field:

1> io:fwrite("|~10w|~n", [{hey, hey, hey}]).
|**********|
ok
2> io:fwrite("|~10s|~n", [io_lib:write({hey, hey, hey})]).
|{hey,hey,h|
3> io:fwrite("|~-10.8s|~n", [io_lib:write({hey, hey, hey})]).
|{hey,hey  |
ok

A list with integers > 255 is considered an error if the Unicode translation modifier is not specified:

4> io:fwrite("~ts~n",[[1024]]).
\x{400}
ok
5> io:fwrite("~s~n",[[1024]]).
** exception error: bad argument
     in function  io:format/3
        called as io:format(<0.53.0>,"~s~n",[[1024]])

w - Writes data with the standard syntax. This is used to output Erlang terms. Atoms are printed within quotes if they contain embedded non-printable characters. Atom characters > 255 are escaped unless the Unicode translation modifier (t) is used. Floats are printed accurately as the shortest, correctly rounded string.

p - Writes the data with standard syntax in the same way as ~w, but breaks terms whose printed representation is longer than one line into many lines and indents each line sensibly. Left-justification is not supported. It also tries to detect flat lists of printable characters and output these as strings. For example:

1> T = [{attributes,[[{id,age,1.50000},{mode,explicit},
{typename,"INTEGER"}], [{id,cho},{mode,explicit},{typename,'Cho'}]]},
{typename,'Person'},{tag,{'PRIVATE',3}},{mode,implicit}].
...
2> io:fwrite("~w~n", [T]).
[{attributes,[[{id,age,1.5},{mode,explicit},{typename,
[73,78,84,69,71,69,82]}],[{id,cho},{mode,explicit},{typena
me,'Cho'}]]},{typename,'Person'},{tag,{'PRIVATE',3}},{mode
,implicit}]
ok
3> io:fwrite("~62p~n", [T]).
[{attributes,[[{id,age,1.5},
               {mode,explicit},
               {typename,"INTEGER"}],
              [{id,cho},{mode,explicit},{typename,'Cho'}]]},
 {typename,'Person'},
 {tag,{'PRIVATE',3}},
 {mode,implicit}]
ok

The field width specifies the maximum line length. It defaults to 80. The precision specifies the initial indentation of the term. It defaults to the number of characters printed on this line in the same call to write/1 or format/1,2,3. For example, using T above:

4> io:fwrite("Here T = ~62p~n", [T]).
Here T = [{attributes,[[{id,age,1.5},
                        {mode,explicit},
                        {typename,"INTEGER"}],
                       [{id,cho},
                        {mode,explicit},
                        {typename,'Cho'}]]},
          {typename,'Person'},
          {tag,{'PRIVATE',3}},
          {mode,implicit}]
ok

As from Erlang/OTP 21.0, a field width of value 0 can be used for specifying that a line is infinitely long, which means that no line breaks are inserted. For example:

5> io:fwrite("~0p~n", [lists:seq(1, 30)]).
[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30]
ok

When the modifier l is specified, no detection of printable character lists takes place, for example:

6> S = [{a,"a"}, {b, "b"}],
   io:fwrite("~15p~n", [S]).
[{a,"a"},
 {b,"b"}]
ok
7> io:fwrite("~15lp~n", [S]).
[{a,[97]},
 {b,[98]}]
ok

The Unicode translation modifier t specifies how to treat characters outside the Latin-1 range of codepoints, in atoms, strings, and binaries. For example, printing an atom containing a character > 255:

8> io:fwrite("~p~n",[list_to_atom([1024])]).
'\x{400}'
ok
9> io:fwrite("~tp~n",[list_to_atom([1024])]).
'Ѐ'
ok

By default, Erlang only detects lists of characters in the Latin-1 range as strings, but the +pc unicode flag can be used to change this (see printable_range/0 for details). For example:

10> io:fwrite("~p~n",[[214]]).
"Ö"
ok
11> io:fwrite("~p~n",[[1024]]).
[1024]
ok
12> io:fwrite("~tp~n",[[1024]]).
[1024]
ok

but if Erlang was started with +pc unicode:

13> io:fwrite("~p~n",[[1024]]).
[1024]
ok
14> io:fwrite("~tp~n",[[1024]]).
"Ѐ"
ok

Similarly, binaries that look like UTF-8 encoded strings are output with the binary string syntax if the t modifier is specified:

15> io:fwrite("~p~n", [<<208,128>>]).
<<208,128>>
ok
16> io:fwrite("~tp~n", [<<208,128>>]).
<<"Ѐ"/utf8>>
ok
17> io:fwrite("~tp~n", [<<128,128>>]).
<<128,128>>
ok

W - Writes data in the same way as ~w, but takes an extra argument that is the maximum depth to which terms are printed. Anything below this depth is replaced with .... For example, using T above:

8> io:fwrite("~W~n", [T,9]).
[{attributes,[[{id,age,1.5},{mode,explicit},{typename,...}],
[{id,cho},{mode,...},{...}]]},{typename,'Person'},
{tag,{'PRIVATE',3}},{mode,implicit}]
ok

If the maximum depth is reached, it cannot be read in the resultant output. Also, the ,... form in a tuple denotes that there are more elements in the tuple but these are below the print depth.

P - Writes data in the same way as ~p, but takes an extra argument that is the maximum depth to which terms are printed. Anything below this depth is replaced with ..., for example:

9> io:fwrite("~62P~n", [T,9]).
[{attributes,[[{id,age,1.5},{mode,explicit},{typename,...}],
              [{id,cho},{mode,...},{...}]]},
 {typename,'Person'},
 {tag,{'PRIVATE',3}},
 {mode,implicit}]
ok

B - Writes an integer in base 2-36, the default base is 10. A leading dash is printed for negative integers.

The precision field selects base, for example:

1> io:fwrite("~.16B~n", [31]).
1F
ok
2> io:fwrite("~.2B~n", [-19]).
-10011
ok
3> io:fwrite("~.36B~n", [5*36+35]).
5Z
ok

X - Like B, but takes an extra argument that is a prefix to insert before the number, but after the leading dash, if any.

The prefix can be a possibly deep list of characters or an atom. Example:

1> io:fwrite("~X~n", [31,"10#"]).
10#31
ok
2> io:fwrite("~.16X~n", [-31,"0x"]).
-0x1F
ok

# - Like B, but prints the number with an Erlang style #-separated base prefix. Example:

1> io:fwrite("~.10#~n", [31]).
10#31
ok
2> io:fwrite("~.16#~n", [-31]).
-16#1F
ok

b - Like B, but prints lowercase letters.

x - Like X, but prints lowercase letters.

+ - Like #, but prints lowercase letters.

n - Writes a new line.

i - Ignores the next term.

The function returns:

ok - The formatting succeeded.

If an error occurs, there is no output. Example:

1> io:fwrite("~s ~w ~i ~w ~c ~n",['abc def', 'abc def', {foo, 1},{foo, 1}, 65]).
abc def 'abc def'  {foo,1} A
ok
2> io:fwrite("~s", [65]).
** exception error: bad argument
     in function  io:format/3
        called as io:format(<0.53.0>,"~s","A")

In this example, an attempt was made to output the single character 65 with the aid of the string formatting directive "~s".

get_chars(Prompt, Count)

-spec get_chars(Prompt, Count) -> Data | server_no_data()
                   when
                       Prompt :: prompt(),
                       Count :: non_neg_integer(),
                       Data :: string() | unicode:unicode_binary().

Equivalent to get_chars(standard_io, Prompt, Count).

get_chars(IoDevice, Prompt, Count)

-spec get_chars(IoDevice, Prompt, Count) -> Data | server_no_data()
                   when
                       IoDevice :: device(),
                       Prompt :: prompt(),
                       Count :: non_neg_integer(),
                       Data :: string() | unicode:unicode_binary().

Reads Count characters from IoDevice, prompting it with Prompt.

The function returns:

Data - The input characters. If the I/O device supports Unicode, the data can represent codepoints > 255 (the latin1 range). If the I/O server is set to deliver binaries, they are encoded in UTF-8 (regardless of whether the I/O device supports Unicode). If you want the data to be returned as a latin1 encoded binary you should use file:read/2 instead.

eof - End of file was encountered.

{error, ErrorDescription} - Other (rare) error condition, such as {error, estale} if reading from an NFS file system.

get_line(Prompt)

-spec get_line(Prompt) -> Data | server_no_data()
                  when Prompt :: prompt(), Data :: string() | unicode:unicode_binary().

Equivalent to get_line(standard_io, Prompt).

get_line(IoDevice, Prompt)

-spec get_line(IoDevice, Prompt) -> Data | server_no_data()
                  when
                      IoDevice :: device(),
                      Prompt :: prompt(),
                      Data :: string() | unicode:unicode_binary().

Reads a line from IoDevice, prompting it with Prompt.

The function returns:

Data - The characters in the line terminated by a line feed (or end of file). If the I/O device supports Unicode, the data can represent codepoints > 255 (the latin1 range). If the I/O server is set to deliver binaries, they are encoded in UTF-8 (regardless of if the I/O device supports Unicode). If you want the data to be returned as a latin1 encoded binary you should use file:read_line/1 instead.

eof - End of file was encountered.

{error, ErrorDescription} - Other (rare) error condition, such as {error, estale} if reading from an NFS file system.

get_password()

(since OTP 28.0)

-spec get_password() -> Data | server_no_data() when Data :: string() | unicode:unicode_binary().

Reads a password from user/0. Works just as get_line/2 except that the typed characters are not printed to the terminal.

In order for this function to work, the shell must be in {noshell, raw} mode. See shell:start_interactive/1 for details on what that means.

Example:

#!/usr/bin/env escript
%%! -noshell

main(_) ->
    ok = shell:start_interactive({noshell, raw}),
    try
        io:get_password()
    after
        shell:start_interactive({noshell, cooked})
    end.

getopts()

-spec getopts() -> [getopt()] | {error, Reason} when Reason :: term().

Equivalent to getopts(standard_io).

getopts(IoDevice)

-spec getopts(IoDevice) -> [getopt()] | {error, Reason} when IoDevice :: device(), Reason :: term().

Requests all available options and their current values for a IoDevice.

For example:

1> {ok,F} = file:open("/dev/null",[read]).
{ok,<0.42.0>}
2> io:getopts(F).
[{binary,false},{encoding,latin1}]

Here the file I/O server returns all available options for a file, which are the expected ones, encoding and binary. However, the standard shell has some more options:

3> io:getopts().
[{expand_fun,#Fun<group.0.120017273>},
 {echo,true},
 {binary,false},
 {encoding,unicode},
 {terminal,true},
 {stdout,true},
 {stderr,true},
 {stdin,true}]

This example is, as can be seen, run in an environment where the terminal supports Unicode input and output.

The stdin, stdout and stderr options are read only and indicates whether the stream is a terminal or not. When it is a terminal, most systems that Erlang runs on allows the use of ANSI escape codes to control what the terminal inputs or outputs.

terminal is an alias for stdout.

See setopts/2 for a description of the other options.

nl()

-spec nl() -> ok.

Equivalent to nl(standard_io).

nl(IoDevice)

-spec nl(IoDevice) -> ok when IoDevice :: device().

Writes new line to the standard output (IoDevice).

parse_erl_exprs(Prompt)

-spec parse_erl_exprs(Prompt) -> Result when Prompt :: prompt(), Result :: parse_ret().

Equivalent to parse_erl_exprs(standard_io, Prompt).

parse_erl_exprs(IoDevice, Prompt)

-spec parse_erl_exprs(IoDevice, Prompt) -> Result
                         when IoDevice :: device(), Prompt :: prompt(), Result :: parse_ret().

Equivalent to parse_erl_exprs(IoDevice, Prompt, 1).

parse_erl_exprs(IoDevice, Prompt, StartLocation)

-spec parse_erl_exprs(IoDevice, Prompt, StartLocation) -> Result
                         when
                             IoDevice :: device(),
                             Prompt :: prompt(),
                             StartLocation :: erl_anno:location(),
                             Result :: parse_ret().

Equivalent to parse_erl_exprs(IoDevice, Prompt, StartLocation, []).

parse_erl_exprs(IoDevice, Prompt, StartLocation, Options)

(since OTP R16B)

-spec parse_erl_exprs(IoDevice, Prompt, StartLocation, Options) -> Result
                         when
                             IoDevice :: device(),
                             Prompt :: prompt(),
                             StartLocation :: erl_anno:location(),
                             Options :: erl_scan:options(),
                             Result :: parse_ret().

Reads data from IoDevice, prompting it with Prompt.

Starts reading at location StartLocation. Argument Options is passed on as argument Options of function erl_scan:tokens/4. The data is tokenized and parsed as if it was a sequence of Erlang expressions until a final dot (.) is reached.

The function returns:

{ok, ExprList, EndLocation} - The parsing was successful.

{eof, EndLocation} - End of file was encountered by the tokenizer.

eof - End of file was encountered by the I/O server.

{error, ErrorInfo, ErrorLocation} - An error occurred while tokenizing or parsing.

{error, ErrorDescription} - Other (rare) error condition, such as {error, estale} if reading from an NFS file system.

Example:

25> io:parse_erl_exprs('enter>').
enter>abc(), "hey".
{ok, [{call,1,{atom,1,abc},[]},{string,1,"hey"}],2}
26> io:parse_erl_exprs('enter>').
enter>abc("hey".
{error,{1,erl_parse,["syntax error before: ",["'.'"]]},2}

parse_erl_form(Prompt)

-spec parse_erl_form(Prompt) -> Result when Prompt :: prompt(), Result :: parse_form_ret().

Equivalent to parse_erl_form(standard_io, Prompt).

parse_erl_form(IoDevice, Prompt)

-spec parse_erl_form(IoDevice, Prompt) -> Result
                        when IoDevice :: device(), Prompt :: prompt(), Result :: parse_form_ret().

Equivalent to parse_erl_form(IoDevice, Prompt, 1).

parse_erl_form(IoDevice, Prompt, StartLocation)

-spec parse_erl_form(IoDevice, Prompt, StartLocation) -> Result
                        when
                            IoDevice :: device(),
                            Prompt :: prompt(),
                            StartLocation :: erl_anno:location(),
                            Result :: parse_form_ret().

Equivalent to parse_erl_form(IoDevice, Prompt, StartLocation, []).

parse_erl_form(IoDevice, Prompt, StartLocation, Options)

(since OTP R16B)

-spec parse_erl_form(IoDevice, Prompt, StartLocation, Options) -> Result
                        when
                            IoDevice :: device(),
                            Prompt :: prompt(),
                            StartLocation :: erl_anno:location(),
                            Options :: erl_scan:options(),
                            Result :: parse_form_ret().

Reads data from IoDevice, prompting it with Prompt.

Starts reading at location StartLocation. Argument Options is passed on as argument Options of function erl_scan:tokens/4. The data is tokenized and parsed as if it was an Erlang form (one of the valid Erlang expressions in an Erlang source file) until a final dot (.) is reached.

The function returns:

{ok, AbsForm, EndLocation} - The parsing was successful.

{eof, EndLocation} - End of file was encountered by the tokenizer.

eof - End of file was encountered by the I/O server.

{error, ErrorInfo, ErrorLocation} - An error occurred while tokenizing or parsing.

{error, ErrorDescription} - Other (rare) error condition, such as {error, estale} if reading from an NFS file system.

printable_range()

(since OTP R16B)

-spec printable_range() -> unicode | latin1.

Returns the user-requested range of printable Unicode characters.

The user can request a range of characters that are to be considered printable in heuristic detection of strings by the shell and by the formatting functions. This is done by supplying +pc <range> when starting Erlang.

The only valid values for <range> are latin1 and unicode. latin1 means that only code points < 256 (except control characters, and so on) are considered printable. unicode means that all printable characters in all Unicode character ranges are considered printable by the I/O functions.

By default, Erlang is started so that only the latin1 range of characters indicate that a list of integers is a string.

The simplest way to use the setting is to call io_lib:printable_list/1, which uses the return value of this function to decide if a list is a string of printable characters.

Note

In a future release, this function may return more values and ranges. To avoid compatibility problems, it is recommended to use function io_lib:printable_list/1.

put_chars(CharData)

-spec put_chars(CharData) -> ok when CharData :: unicode:chardata().

Equivalent to put_chars(standard_io, CharData).

put_chars(IoDevice, CharData)

-spec put_chars(IoDevice, CharData) -> ok when IoDevice :: device(), CharData :: unicode:chardata().

Writes the characters of CharData to the IoDevice.

If you want to write latin1 encoded bytes to the IoDevice you should use file:write/2 instead.

read(Prompt)

-spec read(Prompt) -> Result
              when
                  Prompt :: prompt(),
                  Result :: {ok, Term :: term()} | server_no_data() | {error, ErrorInfo},
                  ErrorInfo :: erl_scan:error_info() | erl_parse:error_info().

Equivalent to read(standard_io, Prompt).

read(IoDevice, Prompt)

-spec read(IoDevice, Prompt) -> Result
              when
                  IoDevice :: device(),
                  Prompt :: prompt(),
                  Result :: {ok, Term :: term()} | server_no_data() | {error, ErrorInfo},
                  ErrorInfo :: erl_scan:error_info() | erl_parse:error_info().

Reads a term Term from the standard input (IoDevice), prompting it with Prompt.

The function returns:

{ok, Term} - The parsing was successful.

eof - End of file was encountered.

{error, ErrorInfo} - The parsing failed.

{error, ErrorDescription} - Other (rare) error condition, such as {error, estale} if reading from an NFS file system.

read(IoDevice, Prompt, StartLocation)

-spec read(IoDevice, Prompt, StartLocation) -> Result
              when
                  IoDevice :: device(),
                  Prompt :: prompt(),
                  StartLocation :: erl_anno:location(),
                  Result ::
                      {ok, Term :: term(), EndLocation :: erl_anno:location()} |
                      {eof, EndLocation :: erl_anno:location()} |
                      server_no_data() |
                      {error, ErrorInfo, ErrorLocation :: erl_anno:location()},
                  ErrorInfo :: erl_scan:error_info() | erl_parse:error_info().

Equivalent to read(IoDevice, Prompt, StartLocation, []).

read(IoDevice, Prompt, StartLocation, Options)

(since OTP R16B)

-spec read(IoDevice, Prompt, StartLocation, Options) -> Result
              when
                  IoDevice :: device(),
                  Prompt :: prompt(),
                  StartLocation :: erl_anno:location(),
                  Options :: erl_scan:options(),
                  Result ::
                      {ok, Term :: term(), EndLocation :: erl_anno:location()} |
                      {eof, EndLocation :: erl_anno:location()} |
                      server_no_data() |
                      {error, ErrorInfo, ErrorLocation :: erl_anno:location()},
                  ErrorInfo :: erl_scan:error_info() | erl_parse:error_info().

Reads a term Term from IoDevice, prompting it with Prompt.

Reading starts at location StartLocation. Argument Options is passed on as argument Options of function erl_scan:tokens/4.

The function returns:

{ok, Term, EndLocation} - The parsing was successful.

{eof, EndLocation} - End of file was encountered.

{error, ErrorInfo, ErrorLocation} - The parsing failed.

{error, ErrorDescription} - Other (rare) error condition, such as {error, estale} if reading from an NFS file system.

rows()

-spec rows() -> {ok, pos_integer()} | {error, enotsup}.

Equivalent to rows(standard_io).

rows(IoDevice)

-spec rows(IoDevice) -> {ok, pos_integer()} | {error, enotsup} when IoDevice :: device().

Retrieves the number of rows of IoDevice (that is, the height of a terminal).

The function only succeeds for terminal devices, for all other I/O devices the function returns {error, enotsup}.

scan_erl_exprs(Prompt)

-spec scan_erl_exprs(Prompt) -> Result
                        when Prompt :: prompt(), Result :: erl_scan:tokens_result() | server_no_data().

Equivalent to scan_erl_exprs(standard_io, Prompt).

scan_erl_exprs(Device, Prompt)

-spec scan_erl_exprs(Device, Prompt) -> Result
                        when
                            Device :: device(),
                            Prompt :: prompt(),
                            Result :: erl_scan:tokens_result() | server_no_data().

Equivalent to scan_erl_exprs(Device, Prompt, 1).

scan_erl_exprs(Device, Prompt, StartLocation)

-spec scan_erl_exprs(Device, Prompt, StartLocation) -> Result
                        when
                            Device :: device(),
                            Prompt :: prompt(),
                            StartLocation :: erl_anno:location(),
                            Result :: erl_scan:tokens_result() | server_no_data().

Equivalent to scan_erl_exprs(Device, Prompt, StartLocation, []).

scan_erl_exprs(Device, Prompt, StartLocation, Options)

(since OTP R16B)

-spec scan_erl_exprs(Device, Prompt, StartLocation, Options) -> Result
                        when
                            Device :: device(),
                            Prompt :: prompt(),
                            StartLocation :: erl_anno:location(),
                            Options :: erl_scan:options(),
                            Result :: erl_scan:tokens_result() | server_no_data().

Reads data from IoDevice, prompting it with Prompt.

Reading starts at location StartLocation. Argument Options is passed on as argument Options of function erl_scan:tokens/4. The data is tokenized as if it were a sequence of Erlang expressions until a final dot (.) is reached. This token is also returned.

The function returns:

{ok, Tokens, EndLocation} - The tokenization succeeded.

{eof, EndLocation} - End of file was encountered by the tokenizer.

eof - End of file was encountered by the I/O server.

{error, ErrorInfo, ErrorLocation} - An error occurred while tokenizing.

{error, ErrorDescription} - Other (rare) error condition, such as {error, estale} if reading from an NFS file system.

Example:

23> io:scan_erl_exprs('enter>').
enter>abc(), "hey".
{ok,[{atom,1,abc},{'(',1},{')',1},{',',1},{string,1,"hey"},{dot,1}],2}
24> io:scan_erl_exprs('enter>').
enter>1.0er.
{error,{1,erl_scan,{illegal,float}},2}

scan_erl_form(Prompt)

-spec scan_erl_form(Prompt) -> Result
                       when Prompt :: prompt(), Result :: erl_scan:tokens_result() | server_no_data().

Equivalent to scan_erl_form(standard_io, Prompt).

scan_erl_form(IoDevice, Prompt)

-spec scan_erl_form(IoDevice, Prompt) -> Result
                       when
                           IoDevice :: device(),
                           Prompt :: prompt(),
                           Result :: erl_scan:tokens_result() | server_no_data().

Equivalent to scan_erl_form(IoDevice, Prompt, 1).

scan_erl_form(IoDevice, Prompt, StartLocation)

-spec scan_erl_form(IoDevice, Prompt, StartLocation) -> Result
                       when
                           IoDevice :: device(),
                           Prompt :: prompt(),
                           StartLocation :: erl_anno:location(),
                           Result :: erl_scan:tokens_result() | server_no_data().

Equivalent to scan_erl_form(IoDevice, Prompt, StartLocation, []).

scan_erl_form(IoDevice, Prompt, StartLocation, Options)

(since OTP R16B)

-spec scan_erl_form(IoDevice, Prompt, StartLocation, Options) -> Result
                       when
                           IoDevice :: device(),
                           Prompt :: prompt(),
                           StartLocation :: erl_anno:location(),
                           Options :: erl_scan:options(),
                           Result :: erl_scan:tokens_result() | server_no_data().

Reads data from IoDevice, prompting it with Prompt.

Starts reading at location StartLocation (1). Argument Options is passed on as argument Options of function erl_scan:tokens/4. The data is tokenized as if it was an Erlang form (one of the valid Erlang expressions in an Erlang source file) until a final dot (.) is reached. This last token is also returned.

The return values are the same as for scan_erl_exprs/4.

setopts(Opts)

-spec setopts(Opts) -> ok | {error, Reason} when Opts :: [setopt()], Reason :: term().

Equivalent to setopts(standard_io, Opts).

setopts(IoDevice, Opts)

-spec setopts(IoDevice, Opts) -> ok | {error, Reason}
                 when IoDevice :: device(), Opts :: [setopt()], Reason :: term().

Set options for IoDevice. Possible options and values vary depending on the I/O device.

For a list of supported options and their current values on a specific I/O device, use function getopts/1.

The options and values supported by the OTP I/O devices are as follows:

binary, list, or {binary, boolean()} - If set in binary mode (binary or {binary, true}), the I/O server sends binary data (encoded in UTF-8) as answers to the get_line, get_chars, and, if possible, get_until requests (for details, see section The Erlang I/O Protocol) in the User's Guide). The immediate effect is that get_chars/2,3 and get_line/1,2 return UTF-8 binaries instead of lists of characters for the affected I/O device.

By default, all I/O devices in OTP are set in list mode. However, the I/O functions can handle any of these modes and so should other, user-written, modules behaving as clients to I/O servers.

This option is supported by the standard_io/0, user/0 and file:io_server/0 I/O servers.

{echo, boolean()} - Denotes if the terminal is to echo input. Only supported for the standard shell I/O server (group.erl)

{expand_fun, expand_fun()} - Provides a function for tab-completion (expansion) like the Erlang shell. This function is called when the user presses the Tab key. The expansion is active when calling line-reading functions, such as get_line/1,2.

The function is called with the current line, up to the cursor, as a reversed string. It is to return a three-tuple: {yes|no, string(), list()}. The first element gives a beep if no, otherwise the expansion is silent; the second is a string that will be entered at the cursor position; the third is a list of possible expansions. If this list is not empty, it is printed below the current input line. The list of possible expansions can be formatted in different ways to make more advanced expansion suggestions more readable to the user, see edlin_expand:expand/2 for documentation of that.

Trivial example (beep on anything except empty line, which is expanded to "quit"):

fun("") -> {yes, "quit", []};
   (_) -> {no, "", ["quit"]} end

This option is only supported by the standard shell (group.erl).

{line_history, true | false} - Specifies if get_line and get_until I/O requests should be saved in the shell history buffer.

This option is only supported by the standard shell (group.erl).

{log, none | output | input | all} - Tells the I/O server that it should log I/O requests. Requests will be logged at info level to the [otp, kernel, io, input | output | ctrl] domain with the following report:

#{ request := IoRequest, server := pid(), server_name => term() }.

It is important to note that extra care should be taken so that these log reports are not logged to standard_io/0 as that may cause the system to enter an infinite loop.

Example:

1> logger:set_primary_config(level, info).
ok
2> logger:add_handler(stdout, logger_std_h, #{ config => #{ file => "stdout.log" }}).
ok
3> io:setopts(user, [{log, output}]).
ok
4> io:format(user, "Hello~n", []).
Hello
ok
5> file:read_file("stdout.log").
{ok,<<"2024-11-14T09:53:49.275085+01:00 info: <0.89.0> wrote to user, Hello\n">>}

Not all I/O servers support this option. Use io:getopts/1 to check if it is available.

Note

The I/O servers in Erlang/OTP will set the logger domain to [otp, kernel, io, input | output]. The default logger handler will not print this domain, so you need to enable it. This can be done by adding a new filter like this:

logger:add_handler_filter(default, io_domain,
   {fun logger_filters:domain/2, {log,sub,[otp,kernel,io]}}).

{encoding, latin1 | unicode} - Specifies how characters are input or output from or to the I/O device, implying that, for example, a terminal is set to handle Unicode input and output or a file is set to handle UTF-8 data encoding.

The option does not affect how data is returned from the I/O functions or how it is sent in the I/O protocol, it only affects how the I/O device is to handle Unicode characters to the "physical" device.

The standard shell is set for unicode or latin1 encoding when the system is started. The encoding is set with the help of the LANG or LC_CTYPE environment variables on Unix-like system or by other means on other systems. So, the user can input Unicode characters and the I/O device is in {encoding, unicode} mode if the I/O device supports it. The mode can be changed, if the assumption of the runtime system is wrong, by setting this option.

Note

Prior to OTP 26.0, when Erlang was started with the -oldshell or -noshell flags (for example, in an escript), the default encoding for standard_io was set to latin1, meaning that any characters > codepoint 255 were escaped and that input was expected to be plain 8-bit ISO Latin-1. As of OTP 26.0, standard_io always defaults to unicode if its supported, otherwise latin1.

If you want to send raw bytes on standard_io, you now always need to explicitly set the encoding to latin1; otherwise, code points 128-255 will be converted to UTF-8. This is best done by setting the kernel configuration parameter standard_io_encoding to latin1.

Files can also be set in {encoding, unicode}, meaning that data is written and read as UTF-8. More encodings are possible for files, see below.

{encoding, unicode | latin1} is supported by both the standard shell (group.erl including werl on Windows), the 'oldshell' (user.erl), and the file I/O servers.

{encoding, utf8 | utf16 | utf32 | {utf16,big} | {utf16,little} | {utf32,big} | {utf32,little}} - For disk files, the encoding can be set to various UTF variants. This has the effect that data is expected to be read as the specified encoding from the file, and the data is written in the specified encoding to the disk file.

{encoding, utf8} has the same effect as {encoding, unicode} on files.

The extended encodings are only supported on disk files (opened by function file:open/2).

write(Term)

-spec write(Term) -> ok when Term :: term().

Equivalent to write(standard_io, Term).

write(IoDevice, Term)

-spec write(IoDevice, Term) -> ok when IoDevice :: device(), Term :: term().

Writes term Term to IoDevice.

io_lib

I/O library functions.

This module contains functions for converting to and from strings (lists of characters). They are used for implementing the functions in the io module. There is no guarantee that the character lists returned from some of the functions are flat, they can be deep lists. Function lists:flatten/1 can be used for flattening deep lists.

Summary

Types

chars()

An possibly deep list containing only char/0s.

chars_limit()

continuation()

A continuation as returned by fread/3.

depth()

format_spec()

A map describing the contents of a format string.

fread_error()

fread_item()

latin1_string()

Functions

bformat(Format, Data)

Equivalent to bfwrite(Format, Data, []).

bformat(Format, Data, Options)

Equivalent to bfwrite(Format, Data, Options).

bfwrite(Format, Data)

Equivalent to bfwrite(Format, Data, []).

bfwrite(Format, Data, Options)

Binary variant of fwrite/3

build_text(FormatList)

For details, see scan_format/2.

bwrite(Term, Options)

Behaves as write/2 but returns a UTF-8 encoded binary string.

bwrite_string(String, Qoute, InEnc)

Returns the UTF-8 encoded binary String surrounded by Qoute.

char_list(Term)

Returns true if Term is a flat list of characters in the Unicode range, otherwise false.

deep_char_list(Term)

Returns true if Term is a, possibly deep, list of characters in the Unicode range, otherwise false.

deep_latin1_char_list(Term)

Returns true if Term is a, possibly deep, list of characters in the ISO Latin-1 range, otherwise false.

format(Format, Data)

Equivalent to fwrite(Format, Data).

format(Format, Data, Options)

Equivalent to fwrite(Format, Data, Options).

fread(Format, String)

Tries to read String in accordance with the control sequences in Format.

fread(Continuation, CharSpec, Format)

This is the re-entrant formatted reader. The continuation of the first call to the functions must be [].

fwrite(Format, Data)

Returns a character list that represents Data formatted in accordance with Format.

fwrite(Format, Data, Options)

Returns a character list that represents Data formatted in accordance with Format in the same way as fwrite/2 and format/2, but takes an extra argument, a list of options.

indentation(String, StartIndent)

Returns the indentation if String has been printed, starting at StartIndent.

latin1_char_list(Term)

Returns true if Term is a flat list of characters in the ISO Latin-1 range, otherwise false.

nl()

Returns a character list that represents a new line character.

print(Term)

Equivalent to print(Term, 1, 80, -1).

print(Term, Column, LineLength, Depth)

Returns a list of characters that represents Term, but breaks representations longer than one line into many lines and indents each line sensibly.

printable_latin1_list(Term)

Returns true if Term is a flat list of printable ISO Latin-1 characters, otherwise false.

printable_list(Term)

Returns true if Term is a flat list of printable characters, otherwise false.

printable_unicode_list(Term)

Returns true if Term is a flat list of printable Unicode characters, otherwise false.

scan_format(Format, Data)

Returns a list corresponding to the specified format string, where control sequences have been replaced with corresponding tuples. This list can be passed to

unscan_format(FormatList)

For details, see scan_format/2.

write(Term)

Equivalent to write(Term, -1).

write(Term, DepthOrOptions)

Returns a character list that represents Term. Option Depth controls the depth of the structures written.

write_atom(Atom)

Returns the list of characters needed to print atom Atom.

write_atom_as_latin1(Atom)

Returns the list of characters needed to print atom Atom. Non-Latin-1 characters are escaped.

write_char(Char)

Returns the list of characters needed to print a character constant in the Unicode character set.

write_char_as_latin1(Char)

Returns the list of characters needed to print a character constant in the Unicode character set. Non-Latin-1 characters are escaped.

write_latin1_char(Latin1Char)

Returns the list of characters needed to print a character constant in the ISO Latin-1 character set.

write_latin1_string(Latin1String)

Returns the list of characters needed to print Latin1String as a string.

write_string(String)

Returns the list of characters needed to print String as a string.

write_string_as_latin1(String)

Returns the list of characters needed to print String as a string. Non-Latin-1 characters are escaped.

Types

chars()

-type chars() :: [char() | chars()].

An possibly deep list containing only char/0s.

chars_limit()

-type chars_limit() :: integer().

continuation()

-opaque continuation()

A continuation as returned by fread/3.

depth()

(not exported)

-type depth() :: -1 | non_neg_integer().

format_spec()

-type format_spec() ::
          #{control_char := char(),
            args := [any()],
            width := none | integer(),
            adjust := left | right,
            precision := none | integer(),
            pad_char := char(),
            encoding := unicode | latin1,
            strings := boolean(),
            maps_order => maps:iterator_order()}.

A map describing the contents of a format string.

control_char is the type of control sequence: $P, $w, and so on.

args is a list of the arguments used by the control sequence, or an empty list if the control sequence does not take any arguments.

width is the field width.

adjust is the adjustment.

precision is the precision of the printed argument.

pad_char is the padding character.

encoding is set to true if translation modifier t is present.

strings is set to false if modifier l is present.

maps_order is set to undefined by default, ordered if modifier k is present, or reversed or CmpFun if modifier K is present.

fread_error()

-type fread_error() :: atom | based | character | float | format | input | integer | string | unsigned.

fread_item()

-type fread_item() :: string() | atom() | integer() | float().

latin1_string()

-type latin1_string() :: [unicode:latin1_char()].

Functions

bformat(Format, Data)

(since OTP 28.0)

-spec bformat(Format, Data) -> unicode:unicode_binary() when Format :: io:format(), Data :: [term()].

Equivalent to bfwrite(Format, Data, []).

bformat(Format, Data, Options)

(since OTP 28.0)

-spec bformat(Format, Data, Options) -> unicode:unicode_binary()
                 when
                     Format :: io:format(),
                     Data :: [term()],
                     Options :: [Option],
                     Option :: {chars_limit, CharsLimit},
                     CharsLimit :: chars_limit().

Equivalent to bfwrite(Format, Data, Options).

bfwrite(Format, Data)

(since OTP 28.0)

-spec bfwrite(Format, Data) -> unicode:unicode_binary() when Format :: io:format(), Data :: [term()].

Equivalent to bfwrite(Format, Data, []).

bfwrite(Format, Data, Options)

(since OTP 28.0)

-spec bfwrite(Format, Data, Options) -> unicode:unicode_binary()
                 when
                     Format :: io:format(),
                     Data :: [term()],
                     Options :: [Option],
                     Option :: {chars_limit, CharsLimit},
                     CharsLimit :: chars_limit().

Binary variant of fwrite/3

Returns a UTF-8 encoded binary string.

build_text(FormatList)

(since OTP 18.0)

-spec build_text(FormatList) -> chars() when FormatList :: [char() | format_spec()].

For details, see scan_format/2.

bwrite(Term, Options)

(since OTP 28.0)

-spec bwrite(Term, Options) -> unicode:unicode_binary()
                when
                    Term :: term(),
                    Options :: [Option],
                    Option ::
                        {chars_limit, CharsLimit} |
                        {depth, Depth} |
                        {encoding, latin1 | utf8 | unicode} |
                        {maps_order, maps:iterator_order()},
                    CharsLimit :: chars_limit(),
                    Depth :: depth().

Behaves as write/2 but returns a UTF-8 encoded binary string.

bwrite_string(String, Qoute, InEnc)

(since OTP 28.0)

-spec bwrite_string(String, Qoute, InEnc) -> unicode:unicode_binary()
                       when
                           String :: string() | binary(),
                           Qoute :: integer() | [],
                           InEnc :: unicode | latin1.

Returns the UTF-8 encoded binary String surrounded by Qoute.

char_list(Term)

-spec char_list(Term) -> boolean() when Term :: term().

Returns true if Term is a flat list of characters in the Unicode range, otherwise false.

deep_char_list(Term)

-spec deep_char_list(Term) -> boolean() when Term :: term().

Returns true if Term is a, possibly deep, list of characters in the Unicode range, otherwise false.

deep_latin1_char_list(Term)

(since OTP R16B)

-spec deep_latin1_char_list(Term) -> boolean() when Term :: term().

Returns true if Term is a, possibly deep, list of characters in the ISO Latin-1 range, otherwise false.

format(Format, Data)

-spec format(Format, Data) -> chars() when Format :: io:format(), Data :: [term()].

Equivalent to fwrite(Format, Data).

format(Format, Data, Options)

(since OTP 21.0)

-spec format(Format, Data, Options) -> chars()
                when
                    Format :: io:format(),
                    Data :: [term()],
                    Options :: [Option],
                    Option :: {chars_limit, CharsLimit},
                    CharsLimit :: chars_limit().

Equivalent to fwrite(Format, Data, Options).

fread(Format, String)

-spec fread(Format, String) -> Result
               when
                   Format :: string(),
                   String :: string(),
                   Result ::
                       {ok, InputList :: [fread_item()], LeftOverChars :: string()} |
                       {more,
                        RestFormat :: string(),
                        Nchars :: non_neg_integer(),
                        InputStack :: chars()} |
                       {error, {fread, What :: fread_error()}}.

Tries to read String in accordance with the control sequences in Format.

For a detailed description of the available formatting options, see io:fread/3. It is assumed that String contains whole lines.

The function returns:

{ok, InputList, LeftOverChars} - The string was read. InputList is the list of successfully matched and read items, and LeftOverChars are the input characters not used.

{more, RestFormat, Nchars, InputStack} - The string was read, but more input is needed to complete the original format string. RestFormat is the remaining format string, Nchars is the number of characters scanned, and InputStack is the reversed list of inputs matched up to that point.

{error, What} - The read operation failed and parameter What gives a hint about the error.

Example:

3> io_lib:fread("~f~f~f", "15.6 17.3e-6 24.5").
{ok,[15.6,1.73e-5,24.5],[]}

fread(Continuation, CharSpec, Format)

-spec fread(Continuation, CharSpec, Format) -> Return
               when
                   Continuation :: continuation() | [],
                   CharSpec :: string() | eof,
                   Format :: string(),
                   Return ::
                       {more, Continuation1 :: continuation()} |
                       {done, Result, LeftOverChars :: string()},
                   Result ::
                       {ok, InputList :: [fread_item()]} | eof | {error, {fread, What :: fread_error()}}.

This is the re-entrant formatted reader. The continuation of the first call to the functions must be [].

For a complete description of how the re-entrant input scheme works, see Armstrong, Virding, Williams: 'Concurrent Programming in Erlang', Chapter 13.

The function returns:

{done, Result, LeftOverChars} - The input is complete. The result is one of the following:

{ok, InputList} - The string was read. InputList is the list of successfully matched and read items, and LeftOverChars are the remaining characters.

eof - End of file was encountered. LeftOverChars are the input characters not used.

{error, What} - An error occurred and parameter What gives a hint about the error.

{more, Continuation} - More data is required to build a term. Continuation must be passed to fread/3 when more data becomes available.

fwrite(Format, Data)

-spec fwrite(Format, Data) -> chars() when Format :: io:format(), Data :: [term()].

Returns a character list that represents Data formatted in accordance with Format.

For a detailed description of the available formatting options, see io:fwrite/1,2,3. If the format string or argument list contains an error, a fault is generated.

If and only if the Unicode translation modifier is used in the format string (that is, ~ts or ~tc), the resulting list can contain characters beyond the ISO Latin-1 character range (that is, numbers > 255). If so, the result is still an ordinary Erlang string/0, and can well be used in any context where Unicode data is allowed.

fwrite(Format, Data, Options)

(since OTP 21.0)

-spec fwrite(Format, Data, Options) -> chars()
                when
                    Format :: io:format(),
                    Data :: [term()],
                    Options :: [Option],
                    Option :: {chars_limit, CharsLimit},
                    CharsLimit :: chars_limit().

Returns a character list that represents Data formatted in accordance with Format in the same way as fwrite/2 and format/2, but takes an extra argument, a list of options.

Valid option:

{chars_limit, CharsLimit} - A soft limit on the number of characters returned. When the number of characters is reached, remaining structures are replaced by "...". CharsLimit defaults to -1, which means no limit on the number of characters returned.

indentation(String, StartIndent)

-spec indentation(String, StartIndent) -> integer() when String :: string(), StartIndent :: integer().

Returns the indentation if String has been printed, starting at StartIndent.

latin1_char_list(Term)

(since OTP R16B)

-spec latin1_char_list(Term) -> boolean() when Term :: term().

Returns true if Term is a flat list of characters in the ISO Latin-1 range, otherwise false.

nl()

-spec nl() -> string().

Returns a character list that represents a new line character.

print(Term)

-spec print(Term) -> chars() when Term :: term().

Equivalent to print(Term, 1, 80, -1).

print(Term, Column, LineLength, Depth)

-spec print(Term, Column, LineLength, Depth) -> chars()
               when
                   Term :: term(),
                   Column :: non_neg_integer(),
                   LineLength :: non_neg_integer(),
                   Depth :: depth().

Returns a list of characters that represents Term, but breaks representations longer than one line into many lines and indents each line sensibly.

Also tries to detect and output lists of printable characters as strings.

Column is the starting column; defaults to 1.

LineLength is the maximum line length; defaults to 80. The value 0 can be used to specify infinite length, meaning that no line breaks are inserted.

Depth is the maximum print depth; defaults to -1, which means no limitation.

printable_latin1_list(Term)

(since OTP R16B)

-spec printable_latin1_list(Term) -> boolean() when Term :: term().

Returns true if Term is a flat list of printable ISO Latin-1 characters, otherwise false.

printable_list(Term)

-spec printable_list(Term) -> boolean() when Term :: term().

Returns true if Term is a flat list of printable characters, otherwise false.

What is a printable character in this case is determined by startup flag +pc to the Erlang VM; see io:printable_range/0 and erl(1).

printable_unicode_list(Term)

(since OTP R16B)

-spec printable_unicode_list(Term) -> boolean() when Term :: term().

Returns true if Term is a flat list of printable Unicode characters, otherwise false.

scan_format(Format, Data)

(since OTP 18.0)

-spec scan_format(Format, Data) -> FormatList
                     when
                         Format :: io:format(), Data :: [term()], FormatList :: [char() | format_spec()].

Returns a list corresponding to the specified format string, where control sequences have been replaced with corresponding tuples. This list can be passed to:

build_text/1 to have the same effect as format(Format, Args)

unscan_format/1 to get the corresponding pair of Format and Args (with every * and corresponding argument expanded to numeric values)

A typical use of this function is to replace unbounded-size control sequences like ~w and ~p with the depth-limited variants ~W and ~P before formatting to text in, for example, a logger.

unscan_format(FormatList)

(since OTP 18.0)

-spec unscan_format(FormatList) -> {Format, Data}
                       when
                           FormatList :: [char() | format_spec()],
                           Format :: io:format(),
                           Data :: [term()].

For details, see scan_format/2.

write(Term)

-spec write(Term) -> chars() when Term :: term().

Equivalent to write(Term, -1).

write(Term, DepthOrOptions)

-spec write(Term, Depth) -> chars() when Term :: term(), Depth :: depth();
           (Term, Options) -> chars()
               when
                   Term :: term(),
                   Options :: [Option],
                   Option ::
                       {chars_limit, CharsLimit} |
                       {depth, Depth} |
                       {encoding, latin1 | utf8 | unicode} |
                       {maps_order, maps:iterator_order()},
                   CharsLimit :: chars_limit(),
                   Depth :: depth().

Returns a character list that represents Term. Option Depth controls the depth of the structures written.

When the specified depth is reached, everything below this level is replaced by "...".

Depth defaults to -1, which means no limitation. Option CharsLimit puts a soft limit on the number of characters returned. When the number of characters is reached, remaining structures are replaced by "...". CharsLimit defaults to -1, which means no limit on the number of characters returned.

Example:

1> lists:flatten(io_lib:write({1,[2],[3],[4,5],6,7,8,9})).
"{1,[2],[3],[4,5],6,7,8,9}"
2> lists:flatten(io_lib:write({1,[2],[3],[4,5],6,7,8,9}, 5)).
"{1,[2],[3],[...],...}"
3> lists:flatten(io_lib:write({[1,2,3],[4,5],6,7,8,9}, [{chars_limit,20}])).
"{[1,2|...],[4|...],...}"

write_atom(Atom)

-spec write_atom(Atom) -> chars() when Atom :: atom().

Returns the list of characters needed to print atom Atom.

write_atom_as_latin1(Atom)

(since OTP 20.0)

-spec write_atom_as_latin1(Atom) -> latin1_string() when Atom :: atom().

Returns the list of characters needed to print atom Atom. Non-Latin-1 characters are escaped.

write_char(Char)

-spec write_char(Char) -> chars() when Char :: char().

Returns the list of characters needed to print a character constant in the Unicode character set.

write_char_as_latin1(Char)

(since OTP R16B)

-spec write_char_as_latin1(Char) -> latin1_string() when Char :: char().

Returns the list of characters needed to print a character constant in the Unicode character set. Non-Latin-1 characters are escaped.

write_latin1_char(Latin1Char)

(since OTP R16B)

-spec write_latin1_char(Latin1Char) -> latin1_string() when Latin1Char :: unicode:latin1_char().

Returns the list of characters needed to print a character constant in the ISO Latin-1 character set.

write_latin1_string(Latin1String)

(since OTP R16B)

-spec write_latin1_string(Latin1String) -> latin1_string() when Latin1String :: latin1_string().

Returns the list of characters needed to print Latin1String as a string.

write_string(String)

-spec write_string(String) -> chars() when String :: string().

Returns the list of characters needed to print String as a string.

write_string_as_latin1(String)

(since OTP R16B)

-spec write_string_as_latin1(String) -> latin1_string() when String :: string().

Returns the list of characters needed to print String as a string. Non-Latin-1 characters are escaped.

re

This module contains regular expression matching functions for strings and binaries.

The regular expression syntax and semantics resemble that of Perl.

The matching algorithms of the library are based on the PCRE2 library, but not all of the PCRE2 library is interfaced and some parts of the library go beyond what PCRE2 offers. Currently PCRE2 version 10.45 (release date 2024-11-27) is used. The sections of the PCRE2 documentation that are relevant to this module are included here.

Note

The Erlang literal syntax for strings uses the \ (backslash) character as an escape code. You need to escape backslashes in literal strings, both in your code and in the shell, with an extra backslash, that is, "\\" or <<"\\">>.

Since Erlang/OTP 27 you can use verbatim sigils to write literal strings. The example above would be written as ~S"\" or ~B"\".

Perl-Like Regular Expression Syntax

The following sections contain reference material for the regular expressions used by this module. The information is based on the PCRE2 documentation, with changes where this module behaves differently to the PCRE2 library.

Change

Starting with Erlang/OTP 28, the underlying library for handling regular expressions via the re module changes from PCRE to PCRE2. This upgrade brings Erlang's regular expression capabilities more in line with modern standards, particularly Perl, but also introduces several breaking changes and differences in behavior compared to PCRE, see PCRE incompatabilities.

PCRE2 Regular Expression Details

The syntax and semantics of the regular expressions that are supported by PCRE2 are described in detail below. Perl's regular expressions are described in its own documentation, and regular expressions in general are covered in a number of books, some of which have copious examples. Jeffrey Friedl's "Mastering Regular Expressions", published by O'Reilly, covers regular expressions in great detail. This description of PCRE2's regular expressions is intended as reference material.

The reference material is divided into the following sections:

Special Start-of-Pattern Items

Characters and Metacharacters

Backslash

Circumflex and Dollar

Full Stop (Period, Dot) and \N

Matching a Single Code Unit

Square Brackets and Character Classes

Perl Extended Character Classes

Posix Character Classes

Vertical Bar

Internal Option Setting

Groups

Duplicate Group Numbers

Named Capture Groups

Repetition

Atomic Grouping and Possessive Quantifiers

Backreferences

Assertions

Non-Atomic Assertions

Scan Substring Assertions

Script Runs

Conditional Groups

Comments

Recursive Patterns

Groups as Subroutines

Recursion and subroutines with returned capture groups

Oniguruma Subroutine Syntax

Backtracking Control

Special Start-of-Pattern Items

A number of options that can be passed to compile/2 can also be set by special items at the start of a pattern. These are not Perl-compatible, but are provided to make these options accessible to pattern writers who are not able to change the program that processes the pattern. Any number of these items may appear, but they must all be together right at the start of the pattern string, and the letters must be in upper case.

UTF support

Unicode support is basically UTF-8 based. To use Unicode characters, you either call compile/2 or run/3 with option unicode, or the pattern must start with one of these special sequences: (*UTF), which is equivalent to setting unicode. Note that with these instructions, the automatic conversion of lists to UTF-8 is not performed by the re functions. Therefore, using these sequences is not recommended. Add option unicode when running compile/2 instead.

Some applications that allow their users to supply patterns may wish to restrict them to non-UTF data for security reasons. If the never_utf option is passed to compile/2, (*UTF) is not allowed, and its appearance in a pattern causes an error.

Unicode Property Support

Another special sequence that may appear at the start of a pattern is (*UCP). This has the same effect as setting the ucp option: it causes sequences such as \d and \w to use Unicode properties to determine character types, instead of recognizing only characters with codes less than 256 via a lookup table. It also causes upper/lower casing operations to use Unicode properties for characters with code points greater than 127, even when UTF is not set. These behaviours can be changed within the pattern; see Internal Option Setting.

Some applications that allow their users to supply patterns may wish to restrict them for security reasons. If the never_ucp option is passed to compile/2, (*UCP) is not allowed, and its appearance in a pattern causes an error.

Locking Out Empty String Matching

Starting a pattern with (*NOTEMPTY) or (*NOTEMPTY_ATSTART) has the same effect as passing the notempty or notempty_atstart option to whichever matching function is subsequently called to match the pattern. These options lock out the matching of empty strings, either entirely, or only at the start of the subject.

Disabling Start-Up Optimizations

If a pattern starts with (*NO_START_OPT), it has the same effect as setting the no_start_optimize option. This disables several optimizations for quickly reaching "no match" results.

Disabling Automatic Anchoring

If a pattern starts with (*NO_DOTSTAR_ANCHOR). This disables optimizations that apply to patterns whose top-level branches all start with .* (match any number of arbitrary characters).

Setting Match Resource Limits

The caller of run/3 can set a limit on this counter, which therefore limits the amount of computing resource used for a match. The maximum depth of nested backtracking can also be limited; this indirectly restricts the amount of heap memory that is used, but there is also an explicit memory limit that can be set.

These facilities are provided to catch runaway matches that are provoked by patterns with huge matching trees. A common example is a pattern with nested unlimited repeats applied to a long string that does not match. When one of these limits is reached, run/3 gives an error return. The limits can also be set by items at the start of the pattern of the form

(*LIMIT_HEAP=d)
(*LIMIT_MATCH=d)
(*LIMIT_DEPTH=d)

where d is any number of decimal digits. However, the value of the setting must be less than the value set as default for it to have any effect. In other words, the pattern writer can lower the limits set by the programmer, but not raise them. If there is more than one setting of one of these limits, the lower value is used. The heap limit is specified in kibibytes (units of 1024 bytes).

The default value for LIMIT_MATCH and LIMIT_DEPTH is 10,000,000 in the Erlang VM. Notice that the recursion limit does not affect the stack depth of the VM, as PCRE for Erlang is compiled in such a way that the match function never does recursion on the C stack.

Prior to release 10.30, LIMIT_DEPTH was called LIMIT_RECURSION. This name is still recognized for backwards compatibility.

Newline Conventions

PCRE2 supports six different conventions for indicating line breaks in strings: a single CR (carriage return) character, a single LF (linefeed) character, the two-character sequence CRLF, any of the three preceding, any Unicode newline sequence, or the NUL character (binary zero).

It is also possible to specify a newline convention by starting a pattern string with one of the following sequences:

(*CR) - carriage return

(*LF) - linefeed

(*CRLF) - carriage return, followed by linefeed

(*ANYCRLF) - any of the three above

(*ANY) - all Unicode newline sequences

(*NUL) - the NUL character (binary zero)

These override the default and the options given to the compiling function. For example, on a Unix system where LF is the default newline sequence, the pattern

(*CR)a.b

changes the convention to CR. That pattern matches "a\nb" because LF is no longer a newline. If more than one of these settings is present, the last one is used.

The newline convention affects where the circumflex and dollar assertions are true. It also affects the interpretation of the dot metacharacter when dotall is not set, and the behaviour of \N when not followed by an opening brace. However, it does not affect what the \R escape sequence matches. By default, this is any Unicode newline sequence, for Perl compatibility. However, this can be changed; see the next section and the description of \R in Newline Sequences. A change of \R setting can be combined with a change of newline convention.

Specifying What \R Matches

It is possible to restrict \R to match only CR, LF, or CRLF (instead of the complete set of Unicode line endings) by setting the option bsr_anycrlf at compile time. This effect can also be achieved by starting a pattern with (*BSR_ANYCRLF). For completeness, (*BSR_UNICODE) is also recognized, corresponding to bsr_anycrlf.

Characters and Metacharacters

A regular expression is a pattern that is matched against a subject string from left to right. Most characters stand for themselves in a pattern, and match the corresponding characters in the subject. As a trivial example, the pattern

The quick brown fox

matches a portion of a subject string that is identical to itself. When caseless matching is specified (the caseless option or (?i) within the pattern), letters are matched independently of case. Note that there are two ASCII characters, K and S, that, in addition to their lower case ASCII equivalents, are case-equivalent with Unicode U+212A (Kelvin sign) and U+017F (long S) respectively when either unicode or ucp is set.

The power of regular expressions comes from the ability to include wild cards, character classes, alternatives, and repetitions in the pattern. These are encoded in the pattern by the use of metacharacters, which do not stand for themselves but instead are interpreted in some special way.

There are two different sets of metacharacters: those that are recognized anywhere in the pattern except within square brackets, and those that are recognized within square brackets. Outside square brackets, the metacharacters are as follows:

\ - general escape character with several uses

^ - assert start of string (or line, in multiline mode)

$ - assert end of string (or line, in multiline mode)

. - match any character except newline (by default)

[ - start character class definition

| - start of alternative branch

( - start group or control verb

) - end group or control verb

* - 0 or more quantifier

+ - 1 or more quantifier; also "possessive quantifier"

? - 0 or 1 quantifier; also quantifier minimizer

{ - potential start of min/max quantifier

Brace characters { and } are also used to enclose data for constructions such as \g{2} or \k{name}. In almost all uses of braces, space and/or horizontal tab characters that follow { or precede } are allowed and are ignored. In the case of quantifiers, they may also appear before or after the comma.

Part of a pattern that is in square brackets is called a "character class". In a character class the only metacharacters are:

\ - general escape character

^ - negate the class, but only if the first character

- - indicates character range

[ - POSIX character class (if followed by POSIX syntax)

] - terminates the character class

If a pattern is compiled with the extended option, most white space in the pattern, other than in a character class, within a \Q...\E sequence, or between a # outside a character class and the next newline, inclusive, is ignored. An escaping backslash can be used to include a white space or a # character as part of the pattern.

Note

Only these two characters are ignored, not the full set of pattern white space characters that are ignored outside a character class. Option settings can be changed within a pattern; see Internal Option Setting.

The following sections describe the use of each of the metacharacters.

Backslash

The backslash character has several uses. Firstly, if it is followed by a character that is not a digit or a letter, it takes away any special meaning that character may have. This use of backslash as an escape character applies both inside and outside character classes.

For example, if you want to match a * character, you must write \* in the pattern. This escaping action applies whether or not the following character would otherwise be interpreted as a metacharacter, so it is always safe to precede a non-alphanumeric with backslash to specify that it stands for itself. In particular, if you want to match a backslash, you write \\.

Only ASCII digits and letters have any special meaning after a backslash. All other characters (in particular, those whose code points are greater than 127) are treated as literals.

If you want to treat all characters in a sequence as literals, you can do so by putting them between \Q and \E. Note that this includes white space even when the extended option is set so that most other white space is ignored. The behaviour is different from Perl in that $ and @ are handled as literals in \Q...\E sequences in PCRE2, whereas in Perl, $ and @ cause variable interpolation. Also, Perl does "double-quotish backslash interpolation" on any backslashes between \Q and \E which, its documentation says, "may lead to confusing results". PCRE2 treats a backslash between \Q and \E just like any other character. Note the following examples:

Pattern - PCRE2 matches Perl matches

\Qabc$xyz\E - abc$xyz abc followed by the contents of $xyz

\Qabc$xyz\E - abc$xyz abc$xyz

\Qabc\E$\Qxyz\E - abc$xyz abc$xyz

\QA\B\E - A\B A\B

\Q\E - \ \E

The \Q...\E sequence is recognized both inside and outside character classes. An isolated \E that is not preceded by \Q is ignored. If \Q is not followed by \E later in the pattern, the literal interpretation continues to the end of the pattern (that is, \E is assumed at the end). If the isolated \Q is inside a character class, this causes an error, because the character class is then not terminated by a closing square bracket.

Another difference from Perl is that any appearance of \Q or \E inside what might otherwise be a quantifier causes PCRE2 not to recognize the sequence as a quantifier. Perl recognizes a quantifier if (redundantly) either of the numbers is inside \Q...\E, but not if the separating comma is. When not recognized as a quantifier a sequence such as {\Q1\E,2} is treated as the literal string "{1,2}".

Non-Printing Characters

A second use of backslash provides a way of encoding non-printing characters in patterns in a visible manner. There is no restriction on the appearance of non-printing characters in a pattern, but when a pattern is being prepared by text editing, it is often easier to use one of the following escape sequences instead of the binary character it represents. In an ASCII or Unicode environment, these escapes are as follows:

\a - alarm, that is, the BEL character (hex 07)

\cx - "control-x", where x is a non-control ASCII character

\e - escape (hex 1B)

\f - form feed (hex 0C)

\n - linefeed (hex 0A)

\r - carriage return (hex 0D) (but see below)

\t - tab (hex 09)

\0dd - character with octal code 0dd

\ddd - character with octal code ddd, or back reference

\o{ddd..} - character with octal code ddd..

\xhh - character with hex code hh

\x{hhh..} - character with hex code hhh..

\N{U+hhh..} - character with Unicode hex code point hhh..

A description of how back references work is given later, following the discussion of parenthesized groups.

By default, after \x that is not followed by {, one or two hexadecimal digits are read (letters can be in upper or lower case). If the character that follows \x is neither { nor a hexadecimal digit, an error occurs. This is different from Perl's default behaviour, which generates a NUL character, but is in line with the behaviour of Perl's 'strict' mode in re.

Any number of hexadecimal digits may appear between \x{ and }. If a character other than a hexadecimal digit appears between \x{ and }, or if there is no terminating }, an error occurs.

Characters whose code points are less than 256 can be defined by either of the two syntaxes for \x or by an octal sequence. There is no difference in the way they are handled. For example, \xdc is exactly the same as \x{dc} or \334. However, using the braced versions does make such sequences easier to read.

The \N{U+hhh..} escape sequence is recognized only when PCRE2 is operating in UTF mode. Perl also uses \N{name} to specify characters by Unicode name; PCRE2 does not support this. Note that when \N is not followed by an opening brace (curly bracket) it has an entirely different meaning, matching any character that is not a newline.

There are some legacy applications where the escape sequence \r is expected to match a newline.

An error occurs if \c is not followed by a character whose ASCII code point is in the range 32 to 126. The precise effect of \cx is as follows: if x is a lower case letter, it is converted to upper case. Then bit 6 of the character (hex 40) is inverted. Thus \cA to \cZ become hex 01 to hex 1A (A is 41, Z is 5A), but \c{ becomes hex 3B ({ is 7B), and \c; becomes hex 7B (; is 3B). If the code unit following \c has a code point less than 32 or greater than 126, a compile-time error occurs.

Octal Escapes and Back References

The escape \o must be followed by a sequence of octal digits, enclosed in braces. An error occurs if this is not the case. This escape provides a way of specifying character code points as octal numbers greater than 0777, and it also allows octal numbers and backreferences to be unambiguously distinguished.

If braces are not used, after \0 up to two further octal digits are read.

Inside a character class, when a backslash is followed by any octal digit, up to three octal digits are read to generate a code point. Any subsequent digits stand for themselves. The sequences \8 and \9 are treated as the literal characters "8" and "9".

Outside a character class, Perl's handling of a backslash followed by a digit other than 0 is complicated by ambiguity, and Perl has changed over time, causing PCRE2 also to change.

For greater clarity and unambiguity, it is best to avoid following \e by a digit greater than zero. Instead, use \o{...} or \x{...} to specify numerical character code points, and \g{...} to specify backreferences.

Perl Rules for Non-Class Backslash 1-9

All the digits that follow the backslash are read as a decimal number. If the number is less than 10, begins with the digit 8 or 9, or if there are at least that many previous capture groups in the expression, the entire sequence is taken as a back reference. Otherwise, up to three octal digits are read to form a character code. For example:

\040 - is another way of writing an ASCII space

\40 - is the same, provided there are fewer than 40 previous capture groups

\7 - is always a backreference

\11 - might be a backreference, or another way of writing a tab

\011 - is always a tab

\0113 - is a tab followed by the character "3"

\113 - might be a backreference, otherwise the character with octal code 113

\377 - might be a backreference, otherwise the value 255 (decimal)

\81 - is always a backreference

Note that octal values of 100 or greater that are specified using this syntax must not be introduced by a leading zero, because no more than three octal digits are ever read.

Python Rules for Non-Class Backslash 1-9

If there are at least three octal digits after the backslash, exactly three are read as an octal code point number, but the value must be no greater than \377, even in modes where higher code point values are supported. Any subsequent digits stand for themselves. If there are fewer than three octal digits, the sequence is taken as a decimal back reference. Thus, for example, \12 is always a back reference, independent of how many captures there are in the pattern. An error is generated for a reference to a non-existent capturing group.

Constraints on Character Values

Characters that are specified using octal or hexadecimal numbers are limited to certain values, as follows:

8-bit non-UTF mode - no greater than 0xff

8-bit UTF-8 mode - no greater than 0x10ffff and a valid code point

Invalid Unicode code points are all those in the range 0xd800 to 0xdfff (the so-called "surrogate" code points).

Escape Sequences in Character Classes

All the sequences that define a single character value can be used both inside and outside character classes. In addition, inside a character class, \b is interpreted as the backspace character (hex 08).

When not followed by an opening brace, \N is not allowed in a character class. \B, \R, and \X are not special inside a character class. Like other unrecognized alphabetic escape sequences, they cause an error. Outside a character class, these sequences have different meanings.

Unsupported Escape Sequences

In Perl, the sequences \F, \l, \L, \u, and \U are recognized by its string handler and used to modify the case of following characters. By default, PCRE2 does not support these escape sequences in patterns.

Absolute and Relative Backreferences

The sequence \g followed by a signed or unsigned number, optionally enclosed in braces, is an absolute or relative backreference. A named backreference can be coded as \g{name}. Backreferences are discussed later, following the discussion of parenthesized groups.

Absolute and Relative Subroutine Calls

For compatibility with Oniguruma, the non-Perl syntax \g followed by a name or a number enclosed either in angle brackets or single quotes, is an alternative syntax for referencing a capture group as a subroutine. Details are discussed later. Note that \g{...} (Perl syntax) and \g<...> (Oniguruma syntax) are not synonymous. The former is a backreference; the latter is a subroutine call.

Generic Character Types

Another use of backslash is for specifying generic character types:

\d - any decimal digit

\D - any character that is not a decimal digit

\h - any horizontal white space character

\H - any character that is not a horizontal white space character

\N - any character that is not a newline

\s - any white space character

\S - any character that is not a white space character

\v - any vertical white space character

\V - any character that is not a vertical white space character

\w - any "word" character

\W - any "non-word" character

The \N escape sequence has the same meaning as the "." metacharacter when dotall is not set, but setting dotall does not change the meaning of \N. Note that when \N is followed by an opening brace it has a different meaning. See Non-Printing Characters. Perl also uses \N{name} to specify characters by Unicode name; PCRE2 does not support this.

Each pair of lower and upper case escape sequences partitions the complete set of characters into two disjoint sets. Any given character matches one, and only one, of each pair. The sequences can appear both inside and outside character classes. They each match one character of the appropriate type. If the current matching point is at the end of the subject string, all of them fail, because there is no character to match.

The default \s characters are HT (9), LF (10), VT (11), FF (12), CR (13), and space (32), which are defined as white space in the "C" locale. This list may vary if locale-specific matching is taking place. For example, in some locales the "non-breaking space" character (\xA0) is recognized as white space, and in others the VT character is not.

A "word" character is an underscore or any character that is a letter or digit. By default, the definition of letters and digits is controlled by PCRE2's low-valued character tables, and may vary if locale-specific matching is taking place. For example, in a French locale such as "fr_FR" in Unix-like systems, or "french" in Windows, some character codes greater than 127 are used for accented letters, and these are then matched by \w. The use of locales with Unicode is discouraged.

By default, characters whose code points are greater than 127 never match \d, \s, or \w, and always match \D, \S, and \W, although this may be different for characters in the range 128-255 when locale-specific matching is happening. These escape sequences retain their original meanings from before Unicode support was available, mainly for efficiency reasons. If the ucp option is set, the behaviour is changed so that Unicode properties are used to determine character types, as follows:

\d - Any character that matches \p{Nd} (decimal digit)

\s - Any character that matches \p{Z} or \h or \v

\w - Any character that matches \p{L}, \p{N}, \p{Mn}, or \p{Pc}

The addition of \p{Mn} (non-spacing mark) and the replacement of an explicit test for underscore with a test for \p{Pc} (connector punctuation) happened in PCRE2 release 10.43. This brings PCRE2 into line with Perl.

The upper case escapes match the inverse sets of characters. Note that \d matches only decimal digits, whereas \w matches any Unicode digit, as well as other character categories. Note also that ucp affects \b, and \B because they are defined in terms of \w and \W. Matching these sequences is noticeably slower when ucp is set.

The effect of ucp on any one of these escape sequences can be negated by the options within a pattern by means of an Internal Option Setting.

The sequences \h, \H, \v, and \V, in contrast to the other sequences, which match only ASCII characters by default, always match a specific list of code points, whether or not ucp is set. The horizontal space characters are:

U+0009 - Horizontal tab (HT)

U+0020 - Space

U+00A0 - Non-break space

U+1680 - Ogham space mark

U+180E - Mongolian vowel separator

U+2000 - En quad

U+2001 - Em quad

U+2002 - En space

U+2003 - Em space

U+2004 - Three-per-em space

U+2005 - Four-per-em space

U+2006 - Six-per-em space

U+2007 - Figure space

U+2008 - Punctuation space

U+2009 - Thin space

U+200A - Hair space

U+202F - Narrow no-break space

U+205F - Medium mathematical space

U+3000 - Ideographic space

The vertical space characters are:

U+000A - Linefeed (LF)

U+000B - Vertical tab (VT)

U+000C - Form feed (FF)

U+000D - Carriage return (CR)

U+0085 - Next line (NEL)

U+2028 - Line separator

U+2029 - Paragraph separator

In 8-bit, non-UTF-8 mode, only the characters with code points less than 256 are relevant.

Newline Sequences

Outside a character class, by default, the escape sequence \R matches any Unicode newline sequence. In 8-bit non-UTF-8 mode \R is equivalent to the following:

(?>\r\n|\n|\x0b|\f|\r|\x85)

This is an example of an "atomic group", details of which are given below. This particular group matches either the two-character sequence CR followed by LF, or one of the single characters LF (linefeed, U+000A), VT (vertical tab, U+000B), FF (form feed, U+000C), CR (carriage return, U+000D), or NEL (next line, U+0085). Because this is an atomic group, the two-character sequence is treated as a single unit that cannot be split.

In other modes, two additional characters whose code points are greater than 255 are added: LS (line separator, U+2028) and PS (paragraph separator, U+2029). Unicode support is not needed for these characters to be recognized.

It is possible to restrict \R to match only CR, LF, or CRLF (instead of the complete set of Unicode line endings) by setting the option bsr_anycrlf at compile time. (BSR is an abbreviation for "backslash R".) This can be made the default when PCRE2 is built; if this is the case, the other behaviour can be requested via the bsr_anycrlf option. It is also possible to specify these settings by starting a pattern string with one of the following sequences:

(*BSR_ANYCRLF) - CR, LF, or CRLF only

(*BSR_UNICODE) - any Unicode newline sequence

These override the default and the options given to the compiling function. Note that these special settings, which are not Perl-compatible, are recognized only at the very start of a pattern, and that they must be in upper case. If more than one of them is present, the last one is used. They can be combined with a change of newline convention; for example, a pattern can start with:

(*ANY)(*BSR_ANYCRLF)

They can also be combined with the (*UTF) or (*UCP) special sequences. Inside a character class, \R is treated as an unrecognized escape sequence, and causes an error.

Unicode Character Properties

When PCRE2 is built with Unicode support (the default), three additional escape sequences that match characters with specific properties are available. They can be used in any mode, though in 8-bit non-UTF modes these sequences are of course limited to testing characters whose code points are less than U+0100.

Matching characters by Unicode property is not fast, because PCRE2 has to do a multistage table lookup in order to find a character's property. That is why the traditional escape sequences such as \d and \w do not use Unicode properties in PCRE2 by default, though you can make them do so by setting the ucp option or by starting the pattern with (*UCP).

The extra escape sequences that provide property support are:

\p{_xx_} - a character with the _xx_ property

\P{_xx_} - a character without the _xx_ property

\X - a Unicode extended grapheme cluster

For compatibility with Perl, negation can be specified by including a circumflex between the opening brace and the property. For example, \p{^Lu} is the same as \P{Lu}.

In accordance with Unicode's "loose matching" rules, ASCII white space characters, hyphens, and underscores are ignored in the properties represented by _xx_ above. As well as the space character, ASCII white space can be tab, linefeed, vertical tab, formfeed, or carriage return.

Some properties are specified as a name only; others as a name and a value, separated by a colon or an equals sign. The names and values consist of ASCII letters and digits (with one Perl-specific exception, see below). They are not case sensitive. Note, however, that the escapes themselves, \p and \P, _are_ case sensitive. There are abbreviations for many names. The following examples are all equivalent:

\p{bidiclass=al}
\p{BC=al}
\p{ Bidi_Class : AL }
\p{ Bi-di class = Al }
\P{ ^ Bi-di class = Al }

There is support for Unicode script names, Unicode general category properties, "Any", which matches any character (including newline), Bidi_Class, a number of binary (yes/no) properties, and some special PCRE2 properties (described below). Certain other Perl properties such as "InMusicalSymbols" are not supported by PCRE2. Note that \P{Any} does not match any characters, so always causes a match failure.

Script Properties for \p and \P

There are three different syntax forms for matching a script. Each Unicode character has a basic script and, optionally, a list of other scripts ("Script Extensions") with which it is commonly used. Using the Adlam script as an example, \p{sc:Adlam} matches characters whose basic script is Adlam, whereas \p{scx:Adlam} matches, in addition, characters that have Adlam in their extensions list. The full names "script" and "script extensions" for the property types are recognized and, as for all property specifications, an equals sign is an alternative to the colon. If a script name is given without a property type, for example, \p{Adlam}, it is treated as \p{scx:Adlam}. Perl changed to this interpretation at release 5.26 and PCRE2 changed at release 10.40.

Unassigned characters are assigned the "Unknown" script. Others that are not part of an identified script are lumped together as "Common".

The General Category Property for \p and \P

Each character has exactly one Unicode general category property, specified by a two-letter abbreviation. If only one letter is specified with \p or \P, it includes all the general category properties that start with that letter. In this case, in the absence of negation, the curly brackets in the escape sequence are optional; these two examples have the same effect:

\p{L}
\pL

The following general category property codes are supported:

C - Other

Cc - Control

Cf - Format

Cn - Unassigned

Co - Private use

Cs - Surrogate

L - Letter

Lc - Cased letter

Ll - Lower case letter

Lm - Modifier letter

Lo - Other letter

Lt - Title case letter

Lu - Upper case letter

M - Mark

Mc - Spacing mark

Me - Enclosing mark

Mn - Non-spacing mark

N - Number

Nd - Decimal number

Nl - Letter number

No - Other number

P - Punctuation

Pc - Connector punctuation

Pd - Dash punctuation

Pe - Close punctuation

Pf - Final punctuation

Pi - Initial punctuation

Po - Other punctuation

Ps - Open punctuation

S - Symbol

Sc - Currency symbol

Sk - Modifier symbol

Sm - Mathematical symbol

So - Other symbol

Z - Separator

Zl - Line separator

Zp - Paragraph separator

Zs - Space separator

Perl originally used the name L& for the Lc property. This is still supported by Perl, but discouraged. PCRE2 also still supports it. This property matches any character that has the Lu, Ll, or Lt property, in other words, any letter that is not classified as a modifier or "other". From release 10.45 of PCRE2 the properties Lu, Ll, and Lt are all treated as Lc when case-independent matching is set by the caseless option or (?i) within the pattern. The other properties are not affected by caseless matching.

The Cs (Surrogate) property applies only to characters whose code points are in the range U+D800 to U+DFFF. However, they are not valid in Unicode strings and so cannot be tested by PCRE2 in UTF mode.

The long synonyms for property names that Perl supports (such as \p{Letter}) are not supported by PCRE2, nor is it permitted to prefix any of these properties with "Is".

No character that is in the Unicode table has the Cn (unassigned) property. Instead, this property is assumed for any code point that is not in the Unicode table.

Binary (Yes/No) Properties for \p and \P

Unicode defines a number of binary properties, that is, properties whose only values are true or false. You can obtain a list of those that are recognized by \p and \P, along with their abbreviations, by running this command:

pcre2test -LP

The Bidi_Class Property for \p and \P

\p{Bidi_Class:<class>} - matches a character with the given class

\p{BC:<class>} - matches a character with the given class

The recognized classes are:

AL - Arabic letter

AN - Arabic number

B - paragraph separator

BN - boundary neutral

CS - common separator

EN - European number

ES - European separator

ET - European terminator

FSI - first strong isolate

L - left-to-right

LRE - left-to-right embedding

LRI - left-to-right isolate

LRO - left-to-right override

NSM - non-spacing mark

ON - other neutral

PDF - pop directional format

PDI - pop directional isolate

R - right-to-left

RLE - right-to-left embedding

RLI - right-to-left isolate

RLO - right-to-left override

S - segment separator

WS - white space

As in all property specifications, an equals sign may be used instead of a colon and the class names are case-insensitive. Only the short names listed above are recognized; PCRE2 does not at present support any long alternatives.

Extended Grapheme Clusters

The \X escape matches any number of Unicode characters that form an "extended grapheme cluster", and treats the sequence as an atomic group (see below). Unicode supports various kinds of composite character by giving each character a grapheme breaking property, and having rules that use these properties to define the boundaries of extended grapheme clusters. The rules are defined in Unicode Standard Annex 29, "Unicode Text Segmentation". Unicode 11.0.0 abandoned the use of some previous properties that had been used for emojis. Instead it introduced various emoji-specific properties. PCRE2 uses only the Extended Pictographic property.

\X always matches at least one character. Then it decides whether to add additional characters according to the following rules for ending a cluster:

End at the end of the subject string.

Do not end between CR and LF; otherwise end after any control character.

Do not break Hangul (a Korean script) syllable sequences. Hangul characters are of five types: L, V, T, LV, and LVT. An L character may be followed by an L, V, LV, or LVT character; an LV or V character may be followed by a V or T character; an LVT or T character may be followed only by a T character.

Do not end before extending characters or spacing marks or the zero-width joiner (ZWJ) character. Characters with the "mark" property always have the "extend" grapheme breaking property.

Do not end after prepend characters.

Do not end within emoji modifier sequences or emoji ZWJ (zero-width joiner) sequences. An emoji ZWJ sequence consists of a character with the Extended_Pictographic property, optionally followed by one or more characters with the Extend property, followed by the ZWJ character, followed by another Extended_Pictographic character.

Do not break within emoji flag sequences. That is, do not break between regional indicator (RI) characters if there are an odd number of RI characters before the break point.

Otherwise, end the cluster.

PCRE2's Additional Properties

As well as the standard Unicode properties described above, PCRE2 supports four more that make it possible to convert traditional escape sequences such as \w and \s to use Unicode properties. PCRE2 uses these non-standard, non-Perl properties internally when ucp is set. However, they may also be used explicitly. These properties are:

Xan - Any alphanumeric character

Xps - Any POSIX space character

Xsp - Any Perl space character

Xwd - Any Perl "word" character

Xan matches characters that have either the L (letter) or the N (number) property. Xps matches the characters tab, linefeed, vertical tab, form feed, or carriage return, and any other character that has the Z (separator) property (this includes the space character). Xsp is the same as Xps; in PCRE1 it used to exclude vertical tab, for Perl compatibility, but Perl changed. Xwd matches the same characters as Xan, plus those that match Mn (non-spacing mark) or Pc (connector punctuation, which includes underscore).

There is another non-standard property, Xuc, which matches any character that can be represented by a Universal Character Name in C++ and other programming languages. These are the characters $, @, ` (grave accent), and all characters with Unicode code points greater than or equal to U+00A0, except for the surrogates U+D800 to U+DFFF. Note that most base (ASCII) characters are excluded. (Universal Character Names are of the form \uHHHH or \UHHHHHHHH where H is a hexadecimal digit. Note that the Xuc property does not match these sequences but the characters that they represent.)

Resetting the Match Start

In normal use, the escape sequence \K causes any previously matched characters not to be included in the final matched sequence that is returned. For example, the pattern:

foo\Kbar

matches "foobar", but reports that it has matched "bar". \K does not interact with anchoring in any way. The pattern:

^foo\Kbar

matches only when the subject begins with "foobar" (in single line mode), though it again reports the matched string as "bar". This feature is similar to a lookbehind assertion (described below), but the part of the pattern that precedes \K is not constrained to match a limited number of characters, as is required for a lookbehind assertion. The use of \K does not interfere with the setting of captured substrings. For example, when the pattern

(foo)\Kbar

matches "foobar", the first substring is still set to "foo".

From version 5.32.0 Perl forbids the use of \K in lookaround assertions. From release 10.38 PCRE2 also forbids this by default.

Simple Assertions

The final use of backslash is for certain simple assertions. An assertion specifies a condition that has to be met at a particular point in a match, without consuming any characters from the subject string. The use of groups for more complicated assertions is described below. The backslashed assertions are:

\b - matches at a word boundary

\B - matches when not at a word boundary

\A - matches at the start of the subject

\Z - matches at the end of the subject also matches before a newline at the end of the subject

\z - matches only at the end of the subject

\G - matches at the first matching position in the subject

Inside a character class, \b has a different meaning; it matches the backspace character. If any other of these assertions appears in a character class, an "invalid escape sequence" error is generated.

A word boundary is a position in the subject string where the current character and the previous character do not both match \w or \W (i.e. one matches \w and the other matches \W), or the start or end of the string if the first or last character matches \w, respectively. When PCRE2 is built with Unicode support, the meanings of \w and \W can be changed by setting the ucp option. When this is done, it also affects \b and \B. Neither PCRE2 nor Perl has a separate "start of word" or "end of word" metasequence. However, whatever follows \b normally determines which it is. For example, the fragment \ba matches "a" at the start of a word.

The \A, \Z, and \z assertions differ from the traditional circumflex and dollar (described in the next section) in that they only ever match at the very start and end of the subject string, whatever options are set. Thus, they are independent of multiline mode. These three assertions are not affected by the notbol or noteol options, which affect only the behaviour of the circumflex and dollar metacharacters. However, if the startoffset argument of run/3 is non-zero, indicating that matching is to start at a point other than the beginning of the subject, \A can never match. The difference between \Z and \z is that \Z matches before a newline at the end of the string as well as at the very end, whereas \z matches only at the end.

The \G assertion is true only when the current matching position is at the start point of the matching process, as specified by the startoffset argument of run/3. It differs from \A when the value of startoffset is non-zero. By calling run/3 multiple times with appropriate arguments, you can mimic Perl's /g option, and it is in this kind of implementation where \G can be useful.

Note, however, that PCRE2's implementation of \G, being true at the starting character of the matching process, is subtly different from Perl's, which defines it as true at the end of the previous match. In Perl, these can be different when the previously matched string was empty. Because PCRE2 does just one match at a time, it cannot reproduce this behaviour.

If all the alternatives of a pattern begin with \G, the expression is anchored to the starting match position, and the "anchored" flag is set in the compiled regular expression.

Circumflex and Dollar

The circumflex and dollar metacharacters are zero-width assertions. That is, they test for a particular condition being true without consuming any characters from the subject string. These two metacharacters are concerned with matching the starts and ends of lines. If the newline convention is set so that only the two-character sequence CRLF is recognized as a newline, isolated CR and LF characters are treated as ordinary data characters, and are not recognized as newlines.

Outside a character class, in the default matching mode, the circumflex character is an assertion that is true only if the current matching point is at the start of the subject string. If the startoffset argument of run/3 is non-zero, or if notbol is set, circumflex can never match if the multiline option is unset. Inside a character class, circumflex has an entirely different meaning (see below).

Circumflex need not be the first character of the pattern if a number of alternatives are involved, but it should be the first thing in each alternative in which it appears if the pattern is ever to match that branch. If all possible alternatives start with a circumflex, that is, if the pattern is constrained to match only at the start of the subject, it is said to be an "anchored" pattern. (There are also other constructs that can cause a pattern to be anchored.)

The dollar character is an assertion that is true only if the current matching point is at the end of the subject string, or immediately before a newline at the end of the string (by default), unless noteol is set. Note, however, that it does not actually match the newline. Dollar need not be the last character of the pattern if a number of alternatives are involved, but it should be the last item in any branch in which it appears. Dollar has no special meaning in a character class.

The meaning of dollar can be changed so that it matches only at the very end of the string, by setting the dollar_endonly option at compile time. This does not affect the \Z assertion.

The meanings of the circumflex and dollar metacharacters are changed if the multiline option is set. When this is the case, a dollar character matches before any newlines in the string, as well as at the very end, and a circumflex matches immediately after internal newlines as well as at the start of the subject string. It does not match after a newline that ends the string, for compatibility with Perl.

For example, the pattern /^abc$/ matches the subject string "def\nabc" (where \n represents a newline) in multiline mode, but not otherwise. Consequently, patterns that are anchored in single line mode because all branches start with ^ are not anchored in multiline mode, and a match for circumflex is possible when the startoffset argument of run/3 is non-zero. The dollar_endonly option is ignored if multiline is set.

When the Newline Conventions recognizes the two-character sequence CRLF as a newline, this is preferred, even if the single characters CR and LF are also recognized as newlines. For example, if the newline convention is "any", a multiline mode circumflex matches before "xyz" in the string "abc\r\nxyz" rather than after CR, even though CR on its own is a valid newline. (It also matches at the very start of the string, of course.)

Note that the sequences \A, \Z, and \z can be used to match the start and end of the subject in both modes, and if all branches of a pattern start with \A it is always anchored, whether or not multiline is set.

Full Stop (Period, Dot) and \N

Outside a character class, a dot in the pattern matches any one character in the subject string except (by default) a character that signifies the end of a line. One or more characters may be specified as line terminators (see Newline Conventions).

Dot never matches a single line-ending character. When the two-character sequence CRLF is the only line ending, dot does not match CR if it is immediately followed by LF, but otherwise it matches all characters (including isolated CRs and LFs). When ANYCRLF is selected for line endings, no occurrences of CR of LF match dot. When all Unicode line endings are being recognized, dot does not match CR or LF or any of the other line ending characters.

The behaviour of dot with regard to newlines can be changed. If the dotall option is set, a dot matches any one character, without exception. If the two-character sequence CRLF is present in the subject string, it takes two dots to match it.

The handling of dot is entirely independent of the handling of circumflex and dollar, the only relationship being that they both involve newlines. Dot has no special meaning in a character class.

The escape sequence \N when not followed by an opening brace behaves like a dot, except that it is not affected by the dotall option. In other words, it matches any character except one that signifies the end of a line.

When \N is followed by an opening brace it has a different meaning. See Non-Printing Characters. Perl also uses \N{name} to specify characters by Unicode name; PCRE2 does not support this.

Matching a Single Code Unit

Outside a character class, the escape sequence \C matches any one code unit, whether or not a UTF mode is set. In the 8-bit library, one code unit is one byte. Unlike a dot, \C always matches line-ending characters. The feature is provided in Perl in order to match individual bytes in UTF-8 mode, but it is unclear how it can usefully be used.

Because \C breaks up characters into individual code units, matching one unit with \C in UTF-8 mode means that the rest of the string may start with a malformed UTF character. This has undefined results, because PCRE2 assumes that it is matching character by character in a valid UTF string (by default it checks the subject string's validity at the start of processing).

PCRE2 does not allow \C to appear in lookbehind assertions (described below) in UTF-8 mode, because this would make it impossible to calculate the length of the lookbehind.

In general, the \C escape sequence is best avoided. However, one way of using it that avoids the problem of malformed UTF-8 characters is to use a lookahead to check the length of the next character, as in this pattern, which could be used with a UTF-8 string (ignore white space and line breaks):

(?| (?=[\x00-\x7f])(\C) |
      (?=[\x80-\x{7ff}])(\C)(\C) |
      (?=[\x{800}-\x{ffff}])(\C)(\C)(\C) |
      (?=[\x{10000}-\x{1fffff}])(\C)(\C)(\C)(\C))

In this example, a group that starts with (?| resets the capturing parentheses numbers in each alternative (see Duplicate Group Numbers). The assertions at the start of each branch check the next UTF-8 character for values whose encoding uses 1, 2, 3, or 4 bytes, respectively. The character's individual bytes are then captured by the appropriate number of \C groups.

Square Brackets and Character Classes

An opening square bracket introduces a character class, terminated by a closing square bracket. A closing square bracket on its own is not special by default. If a closing square bracket is required as a member of the class, it should be the first data character in the class (after an initial circumflex, if present) or escaped with a backslash. This means that, by default, an empty class cannot be defined.

A character class matches a single character in the subject. A matched character must be in the set of characters defined by the class, unless the first character in the class definition is a circumflex, in which case the subject character must not be in the set defined by the class. If a circumflex is actually required as a member of the class, ensure it is not the first character, or escape it with a backslash.

For example, the character class [aeiou] matches any lower case English vowel, whereas [^aeiou] matches all other characters. Note that a circumflex is just a convenient notation for specifying the characters that are in the class by enumerating those that are not. A class that starts with a circumflex is not an assertion; it still consumes a character from the subject string, and therefore it fails to match if the current pointer is at the end of the string.

Characters in a class may be specified by their code points using \o, \x, or \N{U+hh..} in the usual way. When caseless matching is set, any letters in a class represent both their upper case and lower case versions, so for example, a caseless [aeiou] matches "A" as well as "a", and a caseless [^aeiou] does not match "A", whereas a caseful version would. Note that there are two ASCII characters, K and S, that, in addition to their lower case ASCII equivalents, are case-equivalent with Unicode U+212A (Kelvin sign) and U+017F (long S) respectively when either unicode or ucp is set.

Characters that might indicate line breaks are never treated in any special way when matching character classes, whatever line-ending sequence is in use, and whatever setting of the dotall and multiline options is used. A class such as [^a] always matches one of these characters.

The generic character type escape sequences \d, \D, \h, \H, \p, \P, \s, \S, \v, \V, \w, and \W may appear in a character class, and add the characters that they match to the class. For example, [\dABCDEF] matches any hexadecimal digit. In UTF modes, the ucp option affects the meanings of \d, \s, \w and their upper case partners, just as it does when they appear outside a character class, as described in Generic character types. The escape sequence \b has a different meaning inside a character class; it matches the backspace character. The sequences \B, \R, and \X are not special inside a character class. Like any other unrecognized escape sequences, they cause an error. The same is true for \N when not followed by an opening brace.

The minus (hyphen) character can be used to specify a range of characters in a character class. For example, [d-m] matches any letter between d and m, inclusive. If a minus character is required in a class, it must be escaped with a backslash or appear in a position where it cannot be interpreted as indicating a range, typically as the first or last character in the class, or immediately after a range. For example, [b-d-z] matches letters in the range b to d, a hyphen character, or z.

Perl treats a hyphen as a literal if it appears before or after a POSIX class (see below) or before or after a character type escape such as \d or \H. However, unless the hyphen is the last character in the class, Perl outputs a warning in its warning mode, as this is most likely a user error. As PCRE2 has no facility for warning, an error is given in these cases.

It is not possible to have the literal character "]" as the end character of a range. A pattern such as [W-]46] is interpreted as a class of two characters ("W" and "-") followed by a literal string "46]", so it would match "W46]" or "-46]". However, if the "]" is escaped with a backslash it is interpreted as the end of a range, so [W-\]46] is interpreted as a class containing a range and two other characters. The octal or hexadecimal representation of "]" can also be used to end a range.

Ranges normally include all code points between the start and end characters, inclusive. They can also be used for code points specified numerically, for example [\000-\037]. Ranges can include any characters that are valid for the current mode. In any UTF mode, the so-called "surrogate" characters (those whose code points lie between 0xd800 and 0xdfff inclusive) may not be specified explicitly by default. However, ranges such as [\x{d7ff}-\x{e000}], which include the surrogates, are always permitted.

If a range that includes letters is used when caseless matching is set, it matches the letters in either case. For example, [W-c] is equivalent to [][\^_`wxyzabc], matched caselessly, and in a non-UTF mode, if character tables for a French locale are in use, [\xc8-\xcb] matches accented E characters in both cases.

A circumflex can conveniently be used with the upper case character types to specify a more restricted set of characters than the matching lower case type. For example, the class [^\W_] matches any letter or digit, but not underscore, whereas [\w] includes underscore. A positive character class should be read as "something OR something OR ..." and a negative class as "NOT something AND NOT something AND NOT ...".

The metacharacters that are recognized in character classes are backslash, hyphen (when it can be interpreted as specifying a range), circumflex (only at the start), and the terminating closing square bracket. An opening square bracket is also special when it can be interpreted as introducing a POSIX class (see "Posix character classes" below), or a special compatibility feature (see Compatibility feature for word boundaries" below). Escaping any non-alphanumeric character in a class turns it into a literal, whether or not it would otherwise be a metacharacter.

Perl Extended Character Classes

From release 10.45 PCRE2 supports Perl's (?[...]) extended character class syntax. This can be used to perform set operations such as intersection on character classes.

The syntax permitted within (?[...]) is quite different to ordinary character classes. Inside the extended class, there is an expression syntax consisting of "atoms", operators, and ordinary parentheses "()" used for grouping. Such classes always have the Perl /xx modifier (PCRE2 option PCRE2_EXTENDED_MORE) turned on within them. This means that literal space and tab characters are ignored everywhere in the class.

The allowed atoms are individual characters specified by escape sequences such as \n or \x{123}, character types such as \d, POSIX classes such as [:alpha:], and nested ordinary (non-extended) character classes. For example, in (?[\d & [...]]) the nested class [...] follows the usual rules for ordinary character classes, in which parentheses are not metacharacters, and character literals and ranges are permitted.

Character literals and ranges may not appear outside a nested ordinary character class because they are not atoms in the extended syntax. The extended syntax does not introduce any additional escape sequences, so (?[\y]) is an unknown escape, as it would be in [\y].

In the extended syntax, ^ does not negate a class (except within an ordinary class nested inside an extended class); it is instead a binary operator.

The binary operators are "&" (intersection), "|" or "+" (union), "-" (subtraction) and "^" (symmetric difference). These are left-associative and "&" has higher (tighter) precedence, while the others have equal lower precedence. The one prefix unary operator is "!" (complement), with highest precedence.

Posix Character Classes

Perl supports the POSIX notation for character classes. This uses names enclosed by [: and :] within the enclosing square brackets. PCRE2 also supports this notation. in both ordinary and extended classes. For example,

[01[:alpha:]%]

matches "0", "1", any alphabetic character, or "%". The supported class names are:

alnum - letters and digits

alpha - letters

ascii - character codes 0-127

blank - space or tab only

cntrl - control characters

digit - decimal digits (same as \d)

graph - printing characters, excluding space

lower - lower case letters

print - printing characters, including space

punct - printing characters, excluding letters and digits and space

space - white space (the same as \s from PCRE2 8.34)

upper - upper case letters

word - "word" characters (same as \w)

xdigit - hexadecimal digits

The default "space" characters are HT (9), LF (10), VT (11), FF (12), CR (13), and space (32). If locale-specific matching is taking place, the list of space characters may be different; there may be fewer or more of them. "Space" and \s match the same set of characters, as do "word" and \w.

The name "word" is a Perl extension, and "blank" is a GNU extension from Perl 5.8. Another Perl extension is negation, which is indicated by a ^ character after the colon. For example,

[12[:^digit:]]

matches "1", "2", or any non-digit. PCRE2 (and Perl) also recognize the POSIX syntax [.ch.] and [=ch=] where "ch" is a "collating element", but these are not supported, and an error is given if they are encountered.

By default, characters with values greater than 127 do not match any of the POSIX character classes, although this may be different for characters in the range 128-255 when locale-specific matching is happening. However, in UCP mode, unless certain options are set (see below), some of the classes are changed so that Unicode character properties are used. This is achieved by replacing POSIX classes with other sequences, as follows:

[:alnum:] - Becomes \p{Xan}

[:alpha:] - Becomes \p{L}

[:blank:] - Becomes \h

[:cntrl:] - Becomes \p{Cc}

[:digit:] - Becomes \p{Nd}

[:lower:] - Becomes \p{Ll}

[:space:] - Becomes \p{Xps}

[:upper:] - Becomes \p{Lu}

[:word:] - Becomes \p{Xwd}

Negated versions, such as [:^alpha:], use \P instead of \p. Four other POSIX classes are handled specially in UCP mode:

[:graph:] - This matches characters that have glyphs that mark the page when printed. In Unicode property terms, it matches all characters with the L, M, N, P, S, or Cf properties, except for:

U+061C - Arabic Letter Mark

U+180E - Mongolian Vowel Separator

U+2066 - U+2069 - Various "isolate"s

[:print:] - This matches the same characters as [:graph:] plus space characters that are not controls, that is, characters with the Zs property.

[:punct:] - This matches all characters that have the Unicode P (punctuation) property, plus those characters with code points less than 256 that have the S (Symbol) property.

[:xdigit:] - In addition to the ASCII hexadecimal digits, this also matches the "fullwidth"

versions of those characters, whose Unicode code points start at U+FF10. This is a change that was made in PCRE2 release 10.43 for Perl compatibility.

The other POSIX classes are unchanged by ucp, and match only characters with code points less than 256.

There are two options that can be used to restrict the POSIX classes to ASCII characters when ucp is set. The option PCRE2_EXTRA_ASCII_DIGIT affects just [:digit:] and [:xdigit:]. Within a pattern, this can be set and unset by (?aT) and (?-aT). The PCRE2_EXTRA_ASCII_POSIX option disables UCP processing for all POSIX classes, including [:digit:] and [:xdigit:]. Within a pattern, (?aP) and (?-aP) set and unset both these options for consistency.

Compatibility Feature for Word Boundaries

In the POSIX.2 compliant library that was included in 4.4BSD Unix, the ugly syntax [[:<:]] and [[:>:]] is used for matching "start of word" and "end of word". PCRE2 treats these items as follows:

[[:<:]]  is converted to  \b(?=\w)
[[:>:]]  is converted to  \b(?<=\w)

Only these exact character sequences are recognized. A sequence such as [a[:<:]b] provokes error for an unrecognized POSIX class name. This support is not compatible with Perl. It is provided to help migrations from other environments, and is best not used in any new patterns. Note that \b matches at the start and the end of a word (see "Simple assertions" above), and in a Perl-style pattern the preceding or following character normally shows which is wanted, without the need for the assertions that are used above in order to give exactly the POSIX behaviour. Note also that the ucp option changes the meaning of \w (and therefore \b) by default, so it also affects these POSIX sequences.

Vertical Bar

Vertical bar characters are used to separate alternative patterns. For example, the pattern

gilbert|sullivan

matches either "gilbert" or "sullivan". Any number of alternatives may appear, and an empty alternative is permitted (matching the empty string). The matching process tries each alternative in turn, from left to right, and the first one that succeeds is used. If the alternatives are within a group (defined below), "succeeds" means matching the rest of the main pattern as well as the alternative in the group.

Internal Option Setting

The settings of several options can be changed within a pattern by a sequence of letters enclosed between "(?" and ")". The option letters are:

i  for `caseless`
m  for `multiline`
n  for PCRE2_NO_AUTO_CAPTURE
s  for `dotall`
x  for `extended`
xx for PCRE2_EXTENDED_MORE

For example, (?im) sets caseless, multiline matching. It is also possible to unset these options by preceding the relevant letters with a hyphen, for example (?-im). The two "extended" options are not independent; unsetting either one cancels the effects of both of them.

A combined setting and unsetting such as (?im-sx), which sets caseless and multiline while unsetting dotall and extended, is also permitted. Only one hyphen may appear in the options string. If a letter appears both before and after the hyphen, the option is unset. An empty options setting "(?)" is allowed. Needless to say, it has no effect.

If the first character following (? is a circumflex, it causes all of the above options to be unset. Letters may follow the circumflex to cause some options to be re-instated, but a hyphen may not appear.

Some PCRE2-specific options can be changed by the same mechanism using these pairs or individual letters:

aD for PCRE2_EXTRA_ASCII_BSD
aS for PCRE2_EXTRA_ASCII_BSS
aW for PCRE2_EXTRA_ASCII_BSW
aP for PCRE2_EXTRA_ASCII_POSIX and PCRE2_EXTRA_ASCII_DIGIT
aT for PCRE2_EXTRA_ASCII_DIGIT
r  for PCRE2_EXTRA_CASELESS_RESTRICT
J  for `dupnames`
U  for `ungreedy`

However, except for 'r', these are not unset by (?^), which is equivalent to (?-imnrsx). If 'a' is not followed by any of the upper case letters shown above, it sets (or unsets) all the ASCII options.

PCRE2_EXTRA_ASCII_DIGIT has no additional effect when PCRE2_EXTRA_ASCII_POSIX is set, but including it in (?aP) means that (?-aP) suppresses all ASCII restrictions for POSIX classes.

When one of these option changes occurs at top level (that is, not inside group parentheses), the change applies until a subsequent change, or the end of the pattern. An option change within a groups affects only that part of the group that follows it. At the end of the group these options are reset to the state they were before the group. For example,

(a(?i)b)c

matches abc and aBc and no other strings (assuming caseless is not set externally). Any changes made in one alternative do carry on into subsequent branches within the same group. For example,

(a(?i)b|c)

matches "ab", "aB", "c", and "C", even though when matching "C" the first branch is abandoned before the option setting. This is because the effects of option settings happen at compile time. There would be some very weird behaviour otherwise.

As a convenient shorthand, if any option settings are required at the start of a non-capturing group (see the next section), the option letters may appear between the "?" and the ":". Thus the two patterns

(?i:saturday|sunday)
(?:(?i)saturday|sunday)

match exactly the same set of strings.

Note

There are other PCRE2-specific options, applying to the whole pattern, which can be set by the application when the compiling function is called. In addition, the pattern can contain special leading sequences such as (*CRLF) to override what the application has set or what has been defaulted. Details are given in Newline Sequences. There are also the (*UTF) and (*UCP) leading sequences that can be used to set UTF and Unicode property modes; they are equivalent to setting the unicode and ucp options, respectively. However, the application can set the never_utf or never_ucp options, which lock out the use of the (*UTF) and (*UCP) sequences.

Groups

Groups are delimited by parentheses (round brackets), which can be nested. Turning part of a pattern into a group does two things:

It localizes a set of alternatives. For example, the pattern

cat(aract|erpillar|)

matches "cataract", "caterpillar", or "cat". Without the parentheses, it would match "cataract", "erpillar" or an empty string.

It creates a "capture group". This means that, when the whole pattern matches, the portion of the subject string that matched the group is passed back to the caller, separately from the portion that matched the whole pattern.

Opening parentheses are counted from left to right (starting from 1) to obtain numbers for capture groups. For example, if the string "the red king" is matched against the pattern

the ((red|white) (king|queen))

the captured substrings are "red king", "red", and "king", and are numbered 1, 2, and 3, respectively.

The fact that plain parentheses fulfil two functions is not always helpful. There are often times when grouping is required without capturing. If an opening parenthesis is followed by a question mark and a colon, the group does not do any capturing, and is not counted when computing the number of any subsequent capture groups. For example, if the string "the white queen" is matched against the pattern the ((?:red|white) (king|queen)) the captured substrings are "white queen" and "queen", and are numbered 1 and

The maximum number of capture groups is 65535.

As a convenient shorthand, if any option settings are required at the start of a non-capturing group, the option letters may appear between the "?" and the ":". Thus the two patterns

(?i:saturday|sunday)
(?:(?i)saturday|sunday)

match exactly the same set of strings. Because alternative branches are tried from left to right, and options are not reset until the end of the group is reached, an option setting in one branch does affect subsequent branches, so the above patterns match "SUNDAY" as well as "Saturday".

Duplicate Group Numbers

Perl 5.10 introduced a feature whereby each alternative in a group uses the same numbers for its capturing parentheses. Such a group starts with (?| and is itself a non-capturing group. For example, consider this pattern:

(?|(Sat)ur|(Sun))day

Because the two alternatives are inside a (?| group, both sets of capturing parentheses are numbered one. Thus, when the pattern matches, you can look at captured substring number one, whichever alternative matched. This construct is useful when you want to capture part, but not all, of one of a number of alternatives. Inside a (?| group, parentheses are numbered as usual, but the number is reset at the start of each branch. The numbers of any capturing parentheses that follow the whole group start after the highest number used in any branch. The following example is taken from the Perl documentation. The numbers underneath show in which buffer the captured content will be stored.

# before  ---------------branch-reset----------- after
/ ( a )  (?| x ( y ) z | (p (q) r) | (t) u (v) ) ( z ) /x
# 1            2         2  3        2     3     4

A backreference to a capture group uses the most recent value that is set for the group. The following pattern matches "abcabc" or "defdef":

/(?|(abc)|(def))\1/

In contrast, a subroutine call to a capture group always refers to the first one in the pattern with the given number. The following pattern matches "abcabc" or "defabc":

/(?|(abc)|(def))(?1)/

A relative reference such as (?-1) is no different: it is just a convenient way of computing an absolute group number.

If a condition test for a group's having matched refers to a non-unique number, the test is true if any group with that number has matched.

An alternative approach to using this "branch reset" feature is to use duplicate named groups, as described in the next section.

Named Capture Groups

Identifying capture groups by number is simple, but it can be very hard to keep track of the numbers in complicated patterns. Furthermore, if an expression is modified, the numbers may change. To help with this difficulty, PCRE2 supports the naming of capture groups. This feature was not added to Perl until release 5.10. Python had the feature earlier, and PCRE1 introduced it at release 4.0, using the Python syntax. PCRE2 supports both the Perl and the Python syntax.

In PCRE2, a capture group can be named in one of three ways: (?<name>...) or (?'name'...) as in Perl, or (?P<name>...) as in Python. Names may be up to 128 code units long. When unicode is not set, they may contain only ASCII alphanumeric characters and underscores, but must start with a non-digit. When unicode is set, the syntax of group names is extended to allow any Unicode letter or Unicode decimal digit. In other words, group names must match one of these patterns:

^[_A-Za-z][_A-Za-z0-9]*\z - when unicode is not set

^[_\p{L}][_\p{L}\p{Nd}]*\z - when unicode is set

References to capture groups from other parts of the pattern, such as backreferences, recursion, and conditions, can all be made by name as well as by number.

Named capture groups are allocated numbers as well as names, exactly as if the names were not present. In both PCRE2 and Perl, capture groups are primarily identified by numbers; any names are just aliases for these numbers. The PCRE2 API provides function calls for extracting the complete name-to-number translation table from a compiled pattern, as well as convenience functions for extracting captured substrings by name.

Warning

When more than one capture group has the same number, as described in the previous section, a name given to one of them applies to all of them. Perl allows identically numbered groups to have different names. Consider this pattern, where there are two capture groups, both numbered 1:

(?|(?<AA>aa)|(?<BB>bb))

Perl allows this, with both names AA and BB as aliases of group 1. Thus, after a successful match, both names yield the same value (either "aa" or "bb").

In an attempt to reduce confusion, PCRE2 does not allow the same group number to be associated with more than one name. The example above provokes a compile-time error. However, there is still scope for confusion. Consider this pattern:

(?|(?<AA>aa)|(bb))

Although the second group number 1 is not explicitly named, the name AA is still an alias for any group 1. Whether the pattern matches "aa" or "bb", a reference by name to group AA yields the matched string.

By default, a name must be unique within a pattern, except that duplicate names are permitted for groups with the same number, for example:

(?|(?<AA>aa)|(?<AA>bb))

The duplicate name constraint can be disabled by setting the dupnames option at compile time, or by the use of (?J) within the pattern, see Internal Option Setting.

Duplicate names can be useful for patterns where only one instance of the named capture group can match. Suppose you want to match the name of a weekday, either as a 3-letter abbreviation or as the full name, and in both cases you want to extract the abbreviation. This pattern (ignoring the line breaks) does the job:

(?J)
(?<DN>Mon|Fri|Sun)(?:day)?|
(?<DN>Tue)(?:sday)?|
(?<DN>Wed)(?:nesday)?|
(?<DN>Thu)(?:rsday)?|
(?<DN>Sat)(?:urday)?

There are five capture groups, but only one is ever set after a match. The convenience functions for extracting the data by name returns the substring for the first (and in this example, the only) group of that name that matched. This saves searching to find which numbered group it was. (An alternative way of solving this problem is to use a "branch reset" group, as described in the previous section.)

If you make a backreference to a non-unique named group from elsewhere in the pattern, the groups to which the name refers are checked in the order in which they appear in the overall pattern. The first one that is set is used for the reference. For example, this pattern matches both "foofoo" and "barbar" but not "foobar" or "barfoo":

(?J)(?:(?<n>foo)|(?<n>bar))\k<n>

If you make a subroutine call to a non-unique named group, the one that corresponds to the first occurrence of the name is used. In the absence of duplicate numbers this is the one with the lowest number.

If you use a named reference in a condition test (see the section about conditions below), either to check whether a capture group has matched, or to check for recursion, all groups with the same name are tested. If the condition is true for any one of them, the overall condition is true. This is the same behaviour as testing by number.

Repetition

Repetition is specified by quantifiers, which may follow any one of these items:

a literal data character
the dot metacharacter
the `\C` escape sequence
the `\R` escape sequence
the `\X` escape sequence
any escape sequence that matches a single character
a character class
a backreference
a parenthesized group (including lookaround assertions)
a subroutine call (recursive or otherwise)

If a quantifier does not follow a repeatable item, an error occurs. The general repetition quantifier specifies a minimum and maximum number of permitted matches by giving two numbers in curly brackets (braces), separated by a comma. The numbers must be less than 65536, and the first must be less than or equal to the second. For example,

z{2,4}

matches "zz", "zzz", or "zzzz". A closing brace on its own is not a special character. If the second number is omitted, but the comma is present, there is no upper limit; if the second number and the comma are both omitted, the quantifier specifies an exact number of required matches. Thus

[aeiou]{3,}

matches at least 3 successive vowels, but may match many more, whereas

\d{8}

matches exactly 8 digits. If the first number is omitted, the lower limit is taken as zero; in this case the upper limit must be present.

X{,4} is interpreted as X{0,4}

This is a change in behaviour that happened in Perl 5.34.0 and PCRE2 10.43. In earlier versions such a sequence was not interpreted as a quantifier. Other regular expression engines may behave either way.

If the characters that follow an opening brace do not match the syntax of a quantifier, the brace is taken as a literal character. In particular, this means that {,} is a literal string of three characters.

Note that not every opening brace is potentially the start of a quantifier because braces are used in other items such as \N{U+345} or \k{name}.

In UTF modes, quantifiers apply to characters rather than to individual code units. Thus, for example, \x{100}{2} matches two characters, each of which is represented by a two-byte sequence in a UTF-8 string. Similarly, \X{3} matches three Unicode extended grapheme clusters, each of which may be several code units long (and they may be of different lengths).

The quantifier {0} is permitted, causing the expression to behave as if the previous item and the quantifier were not present. This may be useful for capture groups that are referenced as subroutines from elsewhere in the pattern (see Defining capture groups for use by reference only). Except for parenthesized groups, items that have a {0} quantifier are omitted from the compiled pattern.

For convenience, the three most common quantifiers have single-character abbreviations:

* - is equivalent to {0,}

+ - is equivalent to {1,}

? - is equivalent to {0,1}

It is possible to construct infinite loops by following a group that can match no characters with a quantifier that has no upper limit, for example:

(a?)*

Earlier versions of Perl and PCRE1 used to give an error at compile time for such patterns. However, because there are cases where this can be useful, such patterns are now accepted, but whenever an iteration of such a group matches no characters, matching moves on to the next item in the pattern instead of repeatedly matching an empty string. This does not prevent backtracking into any of the iterations if a subsequent item fails to match.

By default, quantifiers are "greedy", that is, they match as much as possible (up to the maximum number of permitted repetitions), without causing the rest of the pattern to fail. The classic example of where this gives problems is in trying to match comments in C programs. These appear between /* and */ and within the comment, individual * and / characters may appear. An attempt to match C comments by applying the pattern

/\*.*\*/

to the string

/* first comment */  not comment  /* second comment */

fails, because it matches the entire string owing to the greediness of the .* item. However, if a quantifier is followed by a question mark, it ceases to be greedy, and instead matches the minimum number of times possible, so the pattern

/\*.*?\*/

does the right thing with C comments. The meaning of the various quantifiers is not otherwise changed, just the preferred number of matches. Do not confuse this use of question mark with its use as a quantifier in its own right. Because it has two uses, it can sometimes appear doubled, as in

\d??\d

which matches one digit by preference, but can match two if that is the only way the rest of the pattern matches.

If the ungreedy option is set (an option that is not available in Perl), the quantifiers are not greedy by default, but individual ones can be made greedy by following them with a question mark. In other words, it inverts the default behaviour.

When a parenthesized group is quantified with a minimum repeat count that is greater than 1 or with a limited maximum, more memory is required for the compiled pattern, in proportion to the size of the minimum or maximum.

If a pattern starts with .* or .{0,} and the dotall option (equivalent to Perl's /s) is set, thus allowing the dot to match newlines, the pattern is implicitly anchored, because whatever follows will be tried against every character position in the subject string, so there is no point in retrying the overall match at any position after the first. PCRE2 normally treats such a pattern as though it were preceded by \A.

In cases where it is known that the subject string contains no newlines, it is worth setting dotall in order to obtain this optimization, or alternatively, using ^ to indicate anchoring explicitly.

However, there are some cases where the optimization cannot be used. When .* is inside capturing parentheses that are the subject of a backreference elsewhere in the pattern, a match at the start may fail where a later one succeeds. Consider, for example:

(.*)abc\1

If the subject is "xyz123abc123" the match point is the fourth character. For this reason, such a pattern is not implicitly anchored.

Another case where implicit anchoring is not applied is when the leading .* is inside an atomic group. Once again, a match at the start may fail where a later one succeeds. Consider this pattern:

(?>.*?a)b

It matches "ab" in the subject "aab". The use of the backtracking control verbs (*PRUNE) and (*SKIP) also disable this optimization.

When a capture group is repeated, the value captured is the substring that matched the final iteration. For example, after

(tweedle[dume]{3}\s*)+

has matched "tweedledum tweedledee" the value of the captured substring is "tweedledee". However, if there are nested capture groups, the corresponding captured values may have been set in previous iterations. For example, after

(a|(b))+

matches "aba" the value of the second captured substring is "b".

Atomic Grouping and Possessive Quantifiers

With both maximizing ("greedy") and minimizing ("ungreedy" or "lazy") repetition, failure of what follows normally causes the repeated item to be re-evaluated to see if a different number of repeats allows the rest of the pattern to match. Sometimes it is useful to prevent this, either to change the nature of the match, or to cause it fail earlier than it otherwise might, when the author of the pattern knows there is no point in carrying on.

Consider, for example, the pattern \d+foo when applied to the subject line

123456bar

After matching all 6 digits and then failing to match "foo", the normal action of the matcher is to try again with only 5 digits matching the \d+ item, and then with 4, and so on, before ultimately failing. "Atomic grouping" (a term taken from Jeffrey Friedl's book) provides the means for specifying that once a group has matched, it is not to be re-evaluated in this way.

If we use atomic grouping for the previous example, the matcher gives up immediately on failing to match "foo" the first time. The notation is a kind of special parenthesis, starting with (?> as in this example:

(?>\d+)foo

Perl 5.28 introduced an experimental alphabetic form starting with (* which may be easier to remember:

(*atomic:\d+)foo

This kind of parenthesized group "locks up" the part of the pattern it contains once it has matched, and a failure further into the pattern is prevented from backtracking into it. Backtracking past it to previous items, however, works as normal.

An alternative description is that a group of this type matches exactly the string of characters that an identical standalone pattern would match, if anchored at the current point in the subject string.

Atomic groups are not capture groups. Simple cases such as the above example can be thought of as a maximizing repeat that must swallow everything it can. So, while both \d+ and \d+? are prepared to adjust the number of digits they match in order to make the rest of the pattern match, (?>\d+) can only match an entire sequence of digits.

Atomic groups in general can of course contain arbitrarily complicated expressions, and can be nested. However, when the contents of an atomic group is just a single repeated item, as in the example above, a simpler notation, called a "possessive quantifier" can be used. This consists of an additional + character following a quantifier. Using this notation, the previous example can be rewritten as

\d++foo

Note that a possessive quantifier can be used with an entire group, for example:

(abc|xyz){2,3}+

Possessive quantifiers are always greedy; the setting of the ungreedy option is ignored. They are a convenient notation for the simpler forms of atomic group. However, there is no difference in the meaning of a possessive quantifier and the equivalent atomic group, though there may be a performance difference; possessive quantifiers should be slightly faster.

The possessive quantifier syntax is an extension to the Perl 5.8 syntax. Jeffrey Friedl originated the idea (and the name) in the first edition of his book. Mike McCloskey liked it, so implemented it when he built Sun's Java package, and PCRE1 copied it from there. It found its way into Perl at release 5.10.

PCRE2 has an optimization that automatically "possessifies" certain simple pattern constructs. For example, the sequence A+B is treated as A++B because there is no point in backtracking into a sequence of A's when B must follow. This feature can be disabled by the PCRE2_NO_AUTO_POSSESS option, by starting the pattern with (*NO_AUTO_POSSESS).

When a pattern contains an unlimited repeat inside a group that can itself be repeated an unlimited number of times, the use of an atomic group is the only way to avoid some failing matches taking a very long time indeed. The pattern

(\D+|<\d+>)*[!?]

matches an unlimited number of substrings that either consist of non-digits, or digits enclosed in <>, followed by either ! or ?. When it matches, it runs quickly. However, if it is applied to

aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa

it takes a long time before reporting failure. This is because the string can be divided between the internal \D+ repeat and the external * repeat in a large number of ways, and all have to be tried. (The example uses [!?] rather than a single character at the end, because both PCRE2 and Perl have an optimization that allows for fast failure when a single character is used. They remember the last single character that is required for a match, and fail early if it is not present in the string.) If the pattern is changed so that it uses an atomic group, like this:

((?>\D+)|<\d+>)*[!?]

sequences of non-digits cannot be broken, and failure happens quickly.

Backreferences

Outside a character class, a backslash followed by a digit greater than 0 (and possibly further digits) is a backreference to a capture group earlier (that is, to its left) in the pattern, provided there have been that many previous capture groups.

However, if the decimal number following the backslash is less than 8, it is always taken as a backreference, and causes an error only if there are not that many capture groups in the entire pattern. In other words, the group that is referenced need not be to the left of the reference for numbers less than 8. A "forward backreference" of this type can make sense when a repetition is involved and the group to the right has participated in an earlier iteration.

It is not possible to have a numerical "forward backreference" to a group whose number is 8 or more using this syntax because a sequence such as \50 is interpreted as a character defined in octal. See Non-Printing Characters for further details of the handling of digits following a backslash. Other forms of backreferencing do not suffer from this restriction. In particular, there is no problem when named capture groups are used.

Another way of avoiding the ambiguity inherent in the use of digits following a backslash is to use the \g escape sequence. This escape must be followed by a signed or unsigned number, optionally enclosed in braces. These examples are all identical:

(ring), \1
(ring), \g1
(ring), \g{1}

An unsigned number specifies an absolute reference without the ambiguity that is present in the older syntax. It is also useful when literal digits follow the reference. A signed number is a relative reference. Consider this example:

(abc(def)ghi)\g{-1}

The sequence \g{-1} is a reference to the capture group whose number is one less than the number of the next group to be started, so in this example (where the next group would be numbered 3) is it equivalent to \2, and \g{-2} would be equivalent to \1. Note that if this construct is inside a capture group, that group is included in the count, so in this example \g{-2} also refers to group 1:

(A)(\g{-2}B)

The use of relative references can be helpful in long patterns, and also in patterns that are created by joining together fragments that contain references within themselves.

The sequence \g{+1} is a reference to the next capture group that is started after this item, and \g{+2} refers to the one after that, and so on. This kind of forward reference can be useful in patterns that repeat. Perl does not support the use of + in this way.

A backreference matches whatever actually most recently matched the capture group in the current subject string, rather than anything at all that matches the group (see Groups as Subroutines below for a way of doing that). So the pattern

(sens|respons)e and \1ibility

matches "sense and sensibility" and "response and responsibility", but not "sense and responsibility". If caseful matching is in force at the time of the backreference, the case of letters is relevant. For example,

((?i)rah)\s+\1

matches "rah rah" and "RAH RAH", but not "RAH rah", even though the original capture group is matched caselessly.

There are several different ways of writing backreferences to named capture groups. The .NET syntax is \k{name}, the Python syntax is (?=name), and the original Perl syntax is \k<name> or \k'name'. All of these are now supported by both Perl and PCRE2. Perl 5.10's unified backreference syntax, in which \g can be used for both numeric and named references, is also supported by PCRE2. We could rewrite the above example in any of the following ways:

(?<p1>(?i)rah)\s+\k<p1>
(?'p1'(?i)rah)\s+\k{p1}
(?P<p1>(?i)rah)\s+(?P=p1)
(?<p1>(?i)rah)\s+\g{p1}

A capture group that is referenced by name may appear in the pattern before or after the reference.

There may be more than one backreference to the same group. If a group has not actually been used in a particular match, backreferences to it always fail by default. For example, the pattern

(a|(bc))\2

always fails if it starts to match "a" rather than "bc".

Because there may be many capture groups in a pattern, all digits following a backslash are taken as part of a potential backreference number. If the pattern continues with a digit character, some delimiter must be used to terminate the backreference. If the extended or PCRE2_EXTENDED_MORE option is set, this can be white space. Otherwise, the \g{} syntax or an empty comment (see Comments below) can be used.

Recursive Backreferences

A backreference that occurs inside the group to which it refers fails when the group is first used, so, for example, (a\1) never matches. However, such references can be useful inside repeated groups. For example, the pattern

(a|b\1)+

matches any number of "a"s and also "aba", "ababbaa" etc. At each iteration of the group, the backreference matches the character string corresponding to the previous iteration. In order for this to work, the pattern must be such that the first iteration does not need to match the backreference. This can be done using alternation, as in the example above, or by a quantifier with a minimum of zero.

For versions of PCRE2 less than 10.25, backreferences of this type used to cause the group that they reference to be treated as an atomic group. This restriction no longer applies, and backtracking into such groups can occur as normal.

Assertions

An assertion is a test that does not consume any characters. The test must succeed for the match to continue. The simple assertions coded as \b, \B, \A, \G, \Z, \z, ^ and $ are described above.

More complicated assertions are coded as parenthesized groups. If matching such a group succeeds, matching continues after it, but with the matching position in the subject string reset to what it was before the assertion was processed.

A special kind of assertion, called a "scan substring" assertion, matches a subpattern against a previously captured substring. This is described in Scan Substring Assertions. It is a PCRE2 extension, not compatible with Perl.

The other goup-based assertions are of two kinds: those that look ahead of the current position in the subject string, and those that look behind it, and in each case an assertion may be positive (must match for the assertion to be true) or negative (must not match for the assertion to be true).

The Perl-compatible lookaround assertions are atomic. If an assertion is true, but there is a subsequent matching failure, there is no backtracking into the assertion. However, there are some cases where non-atomic assertions can be useful. PCRE2 has some support for these, described in Non-atomic assertions, but they are not Perl-compatible.

A lookaround assertion may appear as the condition in a conditional groups. In this case, the result of matching the assertion determines which branch of the condition is followed.

Assertion groups are not capture groups. If an assertion contains capture groups within it, these are counted for the purposes of numbering the capture groups in the whole pattern. Within each branch of an assertion, locally captured substrings may be referenced in the usual way. For example, a sequence such as (.)\g{-1} can be used to check that two adjacent characters are the same.

When a branch within an assertion fails to match, any substrings that were captured are discarded (as happens with any pattern branch that fails to match). A negative assertion is true only when all its branches fail to match; this means that no captured substrings are ever retained after a successful negative assertion. When an assertion contains a matching branch, what happens depends on the type of assertion.

For a positive assertion, internally captured substrings in the successful branch are retained, and matching continues with the next pattern item after the assertion. For a negative assertion, a matching branch means that the assertion is not true. If such an assertion is being used as a condition in a conditional groups, captured substrings are retained, because matching continues with the "no" branch of the condition. For other failing negative assertions, control passes to the previous backtracking point, thus discarding any captured strings within the assertion.

Most assertion groups may be repeated; though it makes no sense to assert the same thing several times, the side effect of capturing in positive assertions may occasionally be useful. However, an assertion that forms the condition for a conditional group may not be quantified. PCRE2 used to restrict the repetition of assertions, but from release 10.35 the only restriction is that an unlimited maximum repetition is changed to be one more than the minimum. For example, {3,} is treated as {3,4}.

Alphabetic Assertion Names

Traditionally, symbolic sequences such as (?= and (?<= have been used to specify lookaround assertions. Perl 5.28 introduced some experimental alphabetic alternatives which might be easier to remember. They all start with (* instead of (? and must be written using lower case letters. PCRE2 supports the following synonyms:

(*positive_lookahead:  or (*pla: is the same as (?=
(*negative_lookahead:  or (*nla: is the same as (?!
(*positive_lookbehind: or (*plb: is the same as (?<=
(*negative_lookbehind: or (*nlb: is the same as (?<!

For example, (*pla:foo) is the same assertion as (?=foo). In the following sections, the various assertions are described using the original symbolic forms.

Lookahead Assertions

Lookahead assertions start with (?= for positive assertions and (?! for negative assertions. For example,

\w+(?=;)

matches a word followed by a semicolon, but does not include the semicolon in the match, and

foo(?!bar)

matches any occurrence of "foo" that is not followed by "bar". Note that the apparently similar pattern

(?!foo)bar

does not find an occurrence of "bar" that is preceded by something other than "foo"; it finds any occurrence of "bar" whatsoever, because the assertion (?!foo) is always true when the next three characters are "bar". A lookbehind assertion is needed to achieve the other effect.

If you want to force a matching failure at some point in a pattern, the most convenient way to do it is with (?!) because an empty string always matches, so an assertion that requires there not to be an empty string must always fail. The backtracking control verb (*FAIL) or (*F) is a synonym for (?!).

Lookbehind Assertions

Lookbehind assertions start with (?<= for positive assertions and (?<! for negative assertions. For example,

(?<!foo)bar

does find an occurrence of "bar" that is not preceded by "foo". The contents of a lookbehind assertion are restricted such that there must be a known maximum to the lengths of all the strings it matches. There are two cases:

If every top-level alternative matches a fixed length, for example

(?<=colour|color)

there is a limit of 65535 characters to the lengths, which do not have to be the same, as this example demonstrates. This is the only kind of lookbehind supported by PCRE2 versions earlier than 10.43.

In PCRE2 10.43 and later, run/3 supports lookbehind assertions in which one or more top-level alternatives can match more than one string length, for example

(?<=colou?r)

The maximum matching length for any branch of the lookbehind is limited to a value set by the calling program (default 255 characters). Unlimited repetition (for example \d*) is not supported. In some cases, the escape sequence \K (see above) can be used instead of a lookbehind assertion at the start of a pattern to get round the length limit restriction.

In UTF-8, PCRE2 does not allow the \C escape (which matches a single code unit even in a UTF mode) to appear in lookbehind assertions, because it makes it impossible to calculate the length of the lookbehind. The \X and \R escapes, which can match different numbers of code units, are never permitted in lookbehinds.

"Subroutine" calls (see below) such as (?2) or (?&X) are permitted in lookbehinds, as long as the called capture group matches a limited-length string. However, recursion, that is, a "subroutine" call into a group that is already active, is not supported.

PCRE2 supports backreferences in lookbehinds, but only if certain conditions are met. There must be no use of (?| in the pattern (it creates duplicate group numbers), and if the backreference is by name, the name must be unique. Of course, the referenced group must itself match a limited length substring. The following pattern matches words containing at least two characters that begin and end with the same character:

\b(\w)\w++(?<=\1)

Possessive quantifiers can be used in conjunction with lookbehind assertions to specify efficient matching at the end of subject strings. Consider a simple pattern such as

abcd$

when applied to a long string that does not match. Because matching proceeds from left to right, PCRE2 will look for each "a" in the subject and then see if what follows matches the rest of the pattern. If the pattern is specified as

^.*abcd$

the initial .* matches the entire string at first, but when this fails (because there is no following "a"), it backtracks to match all but the last character, then all but the last two characters, and so on. Once again the search for "a" covers the entire string, from right to left, so we are no better off. However, if the pattern is written as

^.*+(?<=abcd)

there can be no backtracking for the .*+ item because of the possessive quantifier; it can match only the entire string. The subsequent lookbehind assertion does a single test on the last four characters. If it fails, the match fails immediately. For long strings, this approach makes a significant difference to the processing time.

Using Multiple Assertions

Several assertions (of any sort) may occur in succession. For example,

(?<=`\d{3}`)(?<!999)foo

matches "foo" preceded by three digits that are not "999". Notice that each of the assertions is applied independently at the same point in the subject string. First there is a check that the previous three characters are all digits, and then there is a check that the same three characters are not "999". This pattern does not match "foo" preceded by six characters, the first of which are digits and the last three of which are not "999". For example, it doesn't match "123abcfoo". A pattern to do that is

(?<=\d{3}...)(?<!999)foo

This time the first assertion looks at the preceding six characters, checking that the first three are digits, and then the second assertion checks that the preceding three characters are not "999".

Assertions can be nested in any combination. For example,

(?<=(?<!foo)bar)baz

matches an occurrence of "baz" that is preceded by "bar" which in turn is not preceded by "foo", while

(?<=\d{3}(?!999)...)foo

is another pattern that matches "foo" preceded by three digits and any three characters that are not "999".

Non-Atomic Assertions

Traditional lookaround assertions are atomic. That is, if an assertion is true, but there is a subsequent matching failure, there is no backtracking into the assertion. However, there are some cases where non-atomic positive assertions can be useful. PCRE2 provides these using the following syntax:

(*non_atomic_positive_lookahead:  or (*napla: or (?*
(*non_atomic_positive_lookbehind: or (*naplb: or (?<*

Consider the problem of finding the right-most word in a string that also appears earlier in the string, that is, it must appear at least twice in total. This pattern returns the required result as captured substring 1:

^(?x)(*napla: .* \b(\w++)) (?> .*? \b\1\b ){2}

For a subject such as "word1 word2 word3 word2 word3 word4" the result is "word3". How does it work? At the start, ^(?x) anchors the pattern and sets the "x" option, which causes white space (introduced for readability) to be ignored. Inside the assertion, the greedy .* at first consumes the entire string, but then has to backtrack until the rest of the assertion can match a word, which is captured by group 1. In other words, when the assertion first succeeds, it captures the right-most word in the string.

The current matching point is then reset to the start of the subject, and the rest of the pattern match checks for two occurrences of the captured word, using an ungreedy .? to scan from the left. If this succeeds, we are done, but if the last word in the string does not occur twice, this part of the pattern fails. If a traditional atomic lookahead (?= or `(pla:` had been used, the assertion could not be re-entered, and the whole match would fail. The pattern would succeed only if the very last word in the subject was found twice.

Using a non-atomic lookahead, however, means that when the last word does not occur twice in the string, the lookahead can backtrack and find the second-last word, and so on, until either the match succeeds, or all words have been tested.

Two conditions must be met for a non-atomic assertion to be useful: the contents of one or more capturing groups must change after a backtrack into the assertion, and there must be a backreference to a changed group later in the pattern. If this is not the case, the rest of the pattern match fails exactly as before because nothing has changed, so using a non-atomic assertion just wastes resources.

There is one exception to backtracking into a non-atomic assertion. If an (*ACCEPT) control verb is triggered, the assertion succeeds atomically. That is, a subsequent match failure cannot backtrack into the assertion.

Note that assertions that appear as conditions for conditional groups must be atomic.

Scan Substring Assertions

A special kind of assertion, not compatible with Perl, makes it possible to check the contents of a captured substring by matching it with a subpattern. Because this involves capturing.

A scan substring assertion starts with the sequence (*scan_substring: or (*scs: which is followed by a list of substring numbers (absolute or relative) and/or substring names enclosed in single quotes or angle brackets, all within parentheses. The rest of the item is the subpattern that is applied to the substring, as shown in these examples:

(*scan_substring:(1)...)
(*scs:(-2)...)
(*scs:('AB')...)
(*scs:(1,'AB',-2)...)

The list of groups is checked in the order they are given, and it is the contents of the first one that is found to be set that are scanned. When dupnames is set and there are ambiguous group names, all groups with the same name are checked in numerical order. A scan substring assertion fails if none of the groups it references have been set.

The pattern match on the substring is always anchored, that is, it must match from the start of the substring. There is no "bumpalong" if it does not match at the start. The end of the subject is temporarily reset to be the end of the substring, so \Z, \z, and $ will match there. However, the start of the subject is not reset. This means that ^ matches only if the substring is actually at the start of the main subject, but it also means that lookbehind assertions into what precedes the substring are possible.

Here is a very simple example: find a word that contains the rare (in English) sequence of letters "rh" not at the start:

\b(\w++)(*scs:(1).+rh)

The first group captures a word which is then scanned by the second group. This example does not actually need this heavyweight feature; the same match can be achieved with:

\b\w+?rh\w*\eb

When things are more complicated, however, scanning a captured substring can be a useful way to describe the required match. For exmple, there is a rather complicated pattern in the PCRE2 test data that checks an entire subject string for a palindrome, that is, the sequence of letters is the same in both directions. Suppose you want to search for individual words of two or more characters such as "level" that are palindromes:

(\b\w{2,}+\b)(*scs:(1)...palindrome-matching-pattern...)

Within a substring scanning subpattern, references to other groups work as normal. Capturing groups may appear, and will retain their values during ongoing matching if the assertion succeeds.

Script Runs

In concept, a script run is a sequence of characters that are all from the same Unicode script such as Latin or Greek. However, because some scripts are commonly used together, and because some diacritical and other marks are used with multiple scripts, it is not that simple. There is a full description of the rules that PCRE2 uses in the section entitled "Script Runs" in the pcre2unicode documentation.

If part of a pattern is enclosed between (*script_run: or (*sr: and a closing parenthesis, it fails if the sequence of characters that it matches are not a script run. After a failure, normal backtracking occurs. Script runs can be used to detect spoofing attacks using characters that look the same, but are from different scripts. The string "paypal.com" is an infamous example, where the letters could be a mixture of Latin and Cyrillic. This pattern ensures that the matched characters in a sequence of non-spaces that follow white space are a script run:

\s+(*sr:\S+)

To be sure that they are all from the Latin script (for example), a lookahead can be used:

\s+(?=\p{Latin})(*sr:\S+)

This works as long as the first character is expected to be a character in that script, and not (for example) punctuation, which is allowed with any script. If this is not the case, a more creative lookahead is needed. For example, if digits, underscore, and dots are permitted at the start:

\s+(?=[0-9_.]*\p{Latin})(*sr:\S+)

In many cases, backtracking into a script run pattern fragment is not desirable. The script run can employ an atomic group to prevent this. Because this is a common requirement, a shorthand notation is provided by (*atomic_script_run: or (*asr:

(*asr:...) is the same as (*sr:(?>...))

Note that the atomic group is inside the script run. Putting it outside would not prevent backtracking into the script run pattern.

Support for script runs is not available if PCRE2 is compiled without Unicode support. A compile-time error is given if any of the above constructs is encountered.

Warning

The (*ACCEPT) control verb (see below) should not be used within a script run group, because it causes an immediate exit from the group, bypassing the script run checking.

Conditional Groups

It is possible to cause the matching process to obey a pattern fragment conditionally or to choose between two alternative fragments, depending on the result of an assertion, or whether a specific capture group has already been matched. The two possible forms of conditional group are:

(?(condition)yes-pattern)
(?(condition)yes-pattern|no-pattern)

If the condition is satisfied, the yes-pattern is used; otherwise the no-pattern (if present) is used. An absent no-pattern is equivalent to an empty string (it always matches). If there are more than two alternatives in the group, a compile-time error occurs. Each of the two alternatives may itself contain nested groups of any form, including conditional groups; the restriction to two alternatives applies only at the level of the condition itself. This pattern fragment is an example where the alternatives are complex:

(?(1) (A|B|C) |  (D |  (?(2)E|F) |  E) )

There are five kinds of condition: references to capture groups, references to recursion, two pseudo-conditions called DEFINE and VERSION, and assertions.

Checking for a Used Capture Group by Number

If the text between the parentheses consists of a sequence of digits, the condition is true if a capture group of that number has previously matched. If there is more than one capture group with the same number (see Duplicate Group Numbers), the condition is true if any of them have matched. An alternative notation, which is a PCRE2 extension, not supported by Perl, is to precede the digits with a plus or minus sign. In this case, the group number is relative rather than absolute. The most recently opened capture group (which could be enclosing this condition) can be referenced by (?(-1), the next most recent by (?(-2), and so on. Inside loops it can also make sense to refer to subsequent groups. The next capture group to be opened can be referenced as (?(+1), and so on. The value zero in any of these forms is not used; it provokes a compile-time error.

Consider the following pattern, which contains non-significant white space to make it more readable (assume the extended option) and to divide it into three parts for ease of discussion:

( ( )? - [^()]+ (?(1) ) )

The first part matches an optional opening parenthesis, and if that character is present, sets it as the first captured substring. The second part matches one or more characters that are not parentheses. The third part is a conditional group that tests whether or not the first capture group matched. If it did, that is, if subject started with an opening parenthesis, the condition is true, and so the yes-pattern is executed and a closing parenthesis is required. Otherwise, since no-pattern is not present, the conditional group matches nothing. In other words, this pattern matches a sequence of non-parentheses, optionally enclosed in parentheses.

If you were embedding this pattern in a larger one, you could use a relative reference:

...other stuff... ( ( )? - [^()]+ (?(-1) ) ) ...

This makes the fragment independent of the parentheses in the larger pattern.

Checking for a Used Capture Group by Name

Perl uses the syntax (?(<name>)...) or (?('name')...) to test for a used capture group by name. For compatibility with earlier versions of PCRE1, which had this facility before Perl, the syntax (?(name)...) is also recognized. Note, however, that undelimited names consisting of the letter R followed by digits are ambiguous (see the following section). Rewriting the above example to use a named group gives this:

(?<OPEN> ( )? - [^()]+ (?(<OPEN>) ) )

If the name used in a condition of this kind is a duplicate, the test is applied to all groups of the same name, and is true if any one of them has matched.

Checking for Pattern Recursion

"Recursion" in this sense refers to any subroutine-like call from one part of the pattern to another, whether or not it is actually recursive. See Recursive Patterns and Groups as Subroutines for details of recursion and subroutine calls.

If a condition is the string (R), and there is no capture group with the name R, the condition is true if matching is currently in a recursion or subroutine call to the whole pattern or any capture group. If digits follow the letter R, and there is no group with that name, the condition is true if the most recent call is into a group with the given number, which must exist somewhere in the overall pattern. This is a contrived example that is equivalent to a+b:

((?(R1)a+|(?1)b))

However, in both cases, if there is a capture group with a matching name, the condition tests for its being set, as described in the section above, instead of testing for recursion. For example, creating a group with the name R1 by adding (?<R1>) to the above pattern completely changes its meaning.

If a name preceded by ampersand follows the letter R, for example:

(?(R&name)...)

the condition is true if the most recent recursion is into a group of that name (which must exist within the pattern).

This condition does not check the entire recursion stack. It tests only the current level. If the name used in a condition of this kind is a duplicate, the test is applied to all groups of the same name, and is true if any one of them is the most recent recursion.

At "top level", all these recursion test conditions are false.

Defining Capture Groups for Use by Reference Only

If the condition is the string (DEFINE), the condition is always false, even if there is a group with the name DEFINE. In this case, there may be only one alternative in the rest of the conditional group. It is always skipped if control reaches this point in the pattern; the idea of DEFINE is that it can be used to define subroutines that can be referenced from elsewhere. (The use of subroutines is described below.) For example, a pattern to match an IPv4 address such as "192.168.23.245" could be written like this (ignore white space and line breaks):

(?(DEFINE) (?<byte> 2[0-4]\d |  25[0-5] |  1\d\d |  [1-9]?\d) )
\b (?&byte) (\.(?&byte)){3} \b

The first part of the pattern is a DEFINE group inside which another group named "byte" is defined. This matches an individual component of an IPv4 address (a number less than 256). When matching takes place, this part of the pattern is skipped because DEFINE acts like a false condition. The rest of the pattern uses references to the named group to match the four dot-separated components of an IPv4 address, insisting on a word boundary at each end.

Checking the PCRE2 Version

Programs that link with a PCRE2 library can check the version by calling pcre2\_config() with appropriate arguments. Users of applications that do not have access to the underlying code cannot do this. A special "condition" called VERSION exists to allow such users to discover which version of PCRE2 they are dealing with by using this condition to match a string such as "yesno". VERSION must be followed either by "=" or ">=" and a version number. For example:

(?(VERSION>=10.4)yes|no)

This pattern matches "yes" if the PCRE2 version is greater or equal to 10.4, or "no" otherwise. The fractional part of the version number could be ommited.

Assertion Conditions

If the condition is not in any of the above formats, it must be a parenthesized assertion. This may be a positive or negative lookahead or lookbehind assertion. However, it must be a traditional atomic assertion, not one of the non-atomic assertions.

Consider this pattern, again containing non-significant white space, and with the two alternatives on the second line:

(?(?=[^a-z]*[a-z])
\d{2}-[a-z]{3}-\d{2}  |  \d{2}-\d{2}-\d{2} )

The condition is a positive lookahead assertion that matches an optional sequence of non-letters followed by a letter. In other words, it tests for the presence of at least one letter in the subject. If a letter is found, the subject is matched against the first alternative; otherwise it is matched against the second. This pattern matches strings in one of the two forms dd-aaa-dd or dd-dd-dd, where aaa are letters and dd are digits.

When an assertion that is a condition contains capture groups, any capturing that occurs in a matching branch is retained afterwards, for both positive and negative assertions, because matching always continues after the assertion, whether it succeeds or fails. (Compare non-conditional assertions, for which captures are retained only for positive assertions that succeed.)

Comments

There are two ways of including comments in patterns that are processed by PCRE2. In both cases, the start of the comment must not be in a character class, nor in the middle of any other sequence of related characters such as (?: or a group name or number or a Unicode property name. The characters that make up a comment play no part in the pattern matching.

The sequence (?# marks the start of a comment that continues up to the next closing parenthesis. Nested parentheses are not permitted. If the extended or PCRE2_EXTENDED_MORE option is set, an unescaped # character also introduces a comment, which in this case continues to immediately after the next newline character or character sequence in the pattern. Which characters are interpreted as newlines is controlled by an option passed to the compiling function or by a special sequence at the start of the pattern, as described in Newline Conventions. Note that the end of this type of comment is a literal newline sequence in the pattern; escape sequences that happen to represent a newline do not count. For example, consider this pattern when extended is set, and the default newline convention (a single linefeed character) is in force:

abc #comment \n still comment

On encountering the # character, compile/2 skips along, looking for a newline in the pattern. The sequence \n is still literal at this stage, so it does not terminate the comment. Only an actual character with the code value 0x0a (the default newline) does so.

Recursive Patterns

Consider the problem of matching a string in parentheses, allowing for unlimited nested parentheses. Without the use of recursion, the best that can be done is to use a pattern that matches up to some fixed depth of nesting. It is not possible to handle an arbitrary nesting depth.

For some time, Perl has provided a facility that allows regular expressions to recurse (amongst other things). It does this by interpolating Perl code in the expression at run time, and the code can refer to the expression itself. A Perl pattern using code interpolation to solve the parentheses problem can be created like this:

$re =  qr{\( (?: (?>[^()]+) |  (?p{$re}) )* \)}x;

The (?p{...}) item interpolates Perl code at run time, and in this case refers recursively to the pattern in which it appears.

Obviously, PCRE2 cannot support the interpolation of Perl code. Instead, it supports special syntax for recursion of the entire pattern, and also for individual capture group recursion. After its introduction in PCRE1 and Python, this kind of recursion was subsequently introduced into Perl at release 5.10.

A special item that consists of (? followed by a number greater than zero and a closing parenthesis is a recursive subroutine call of the capture group of the given number, provided that it occurs inside that group. (If not, it is a non-recursive subroutine call, which is described in the next section.) The special item (?R) or (?0) is a recursive call of the entire regular expression.

This PCRE2 pattern solves the nested parentheses problem (assume the extended option is set so that white space is ignored):

\( ( [^()]++ | (?R) )* \)

First it matches an opening parenthesis. Then it matches any number of substrings which can either be a sequence of non-parentheses, or a recursive match of the pattern itself (that is, a correctly parenthesized substring). Finally there is a closing parenthesis. Note the use of a possessive quantifier to avoid backtracking into sequences of non-parentheses.

If this were part of a larger pattern, you would not want to recurse the entire pattern, so instead you could use this:

( \( ( [^()]++ | (?1) )* \) )

We have put the pattern into parentheses, and caused the recursion to refer to them instead of the whole pattern.

In a larger pattern, keeping track of parenthesis numbers can be tricky. This is made easier by the use of relative references. Instead of (?1) in the pattern above you can write (?-2) to refer to the second most recently opened parentheses preceding the recursion. In other words, a negative number counts capturing parentheses leftwards from the point at which it is encountered.

Be aware however, that if duplicate capture group numbers are in use, relative references refer to the earliest group with the appropriate number. Consider, for example:

(?|(a)|(b)) (c) (?-2)

The first two capture groups (a) and (b) are both numbered 1, and group (c) is number 2. When the reference (?-2) is encountered, the second most recently opened parentheses has the number 1, but it is the first such group (the (a) group) to which the recursion refers. This would be the same if an absolute reference (?1) was used. In other words, relative references are just a shorthand for computing a group number.

It is also possible to refer to subsequent capture groups, by writing references such as (?+2). However, these cannot be recursive because the reference is not inside the parentheses that are referenced. They are always non-recursive subroutine calls, as described in the next section.

An alternative approach is to use named parentheses. The Perl syntax for this is (?&name); PCRE1's earlier syntax (?P>name) is also supported. We could rewrite the above example as follows:

(?<pn> \( ( [^()]++ | (?&pn) )* \) )

If there is more than one group with the same name, the earliest one is used.

The example pattern that we have been looking at contains nested unlimited repeats, and so the use of a possessive quantifier for matching strings of non-parentheses is important when applying the pattern to strings that do not match. For example, when this pattern is applied to

(aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa()

it yields "no match" quickly. However, if a possessive quantifier is not used, the match runs for a very long time indeed because there are so many different ways the + and * repeats can carve up the subject, and all have to be tested before failure can be reported.

At the end of a match, the values of capturing parentheses are those from the outermost level. If you want to obtain intermediate values, a callout function can be used (see below and the pcre2callout documentation). If the pattern above is matched against

(ab(cd)ef)

the value for the inner capturing parentheses (numbered 2) is "ef", which is the last value taken on at the top level. If a capture group is not matched at the top level, its final captured value is unset, even if it was (temporarily) set at a deeper level during the matching process.

Do not confuse the (?R) item with the condition (R), which tests for recursion. Consider this pattern, which matches text in angle brackets, allowing for arbitrary nesting. Only digits are allowed in nested brackets (that is, when recursing), whereas any characters are permitted at the outer level.

< (?: (?(R) \d++  |  [^<>]*+) |  (?R)) *  >

In this pattern, (?(R) is the start of a conditional group, with two different alternatives for the recursive and non-recursive cases. The (?R) item is the actual recursive call.

Differences in Recursion Processing Between PCRE2 and Perl

Some former differences between PCRE2 and Perl no longer exist.

Before release 10.30, recursion processing in PCRE2 differed from Perl in that a recursive subroutine call was always treated as an atomic group. That is, once it had matched some of the subject string, it was never re-entered, even if it contained untried alternatives and there was a subsequent matching failure. (Historical note: PCRE implemented recursion before Perl did.)

Starting with release 10.30, recursive subroutine calls are no longer treated as atomic. That is, they can be re-entered to try unused alternatives if there is a matching failure later in the pattern. This is now compatible with the way Perl works. If you want a subroutine call to be atomic, you must explicitly enclose it in an atomic group.

Supporting backtracking into recursions simplifies certain types of recursive pattern. For example, this pattern matches palindromic strings:

^((.)(?1)\2|.?)$

The second branch in the group matches a single central character in the palindrome when there are an odd number of characters, or nothing when there are an even number of characters, but in order to work it has to be able to try the second case when the rest of the pattern match fails. If you want to match typical palindromic phrases, the pattern has to ignore all non-word characters, which can be done like this:

^\W*+((.)\W*+(?1)\W*+\2|\W*+.?)\W*+$

If run with the caseless option, this pattern matches phrases such as "A man, a plan, a canal: Panama!". Note the use of the possessive quantifier *+ to avoid backtracking into sequences of non-word characters. Without this, PCRE2 takes a great deal longer (ten times or more) to match typical phrases, and Perl takes so long that you think it has gone into a loop.

Another way in which PCRE2 and Perl used to differ in their recursion processing is in the handling of captured values. Formerly in Perl, when a group was called recursively or as a subroutine (see Groups as Subroutines), it had no access to any values that were captured outside the recursion, whereas in PCRE2 these values can be referenced. Consider this pattern:

^(.)(\1|a(?2))

This pattern matches "bab". The first capturing parentheses match "b", then in the second group, when the backreference \1 fails to match "b", the second alternative matches "a" and then recurses. In the recursion, \1 does now match "b" and so the whole match succeeds. This match used to fail in Perl, but in later versions (I tried 5.024) it now works.

Groups as Subroutines

If the syntax for a recursive group call (either by number or by name) is used outside the parentheses to which it refers, it operates a bit like a subroutine in a programming language. More accurately, PCRE2 treats the referenced group as an independent subpattern which it tries to match at the current matching position. The called group may be defined before or after the reference. A numbered reference can be absolute or relative, as in these examples:

(...(absolute)...)...(?2)...
(...(relative)...)...(?-1)...
(...(?+1)...(relative)...

An earlier example pointed out that the pattern

(sens|respons)e and \1ibility

matches "sense and sensibility" and "response and responsibility", but not "sense and responsibility". If instead the pattern

(sens|respons)e and (?1)ibility

is used, it does match "sense and responsibility" as well as the other two strings. Another example is given in the discussion of DEFINE above.

Like recursions, subroutine calls used to be treated as atomic, but this changed at PCRE2 release 10.30, so backtracking into subroutine calls can now occur. However, any capturing parentheses that are set during the subroutine call revert to their previous values afterwards.

Processing options such as case-independence are fixed when a group is defined, so if it is used as a subroutine, such options cannot be changed for different calls. For example, consider this pattern:

(abc)(?i:(?-1))

It matches "abcabc". It does not match "abcABC" because the change of processing option does not affect the called group.

The behaviour of backtracking control verbs in groups when called as subroutines is described in Backtracking verbs in subroutines.

Recursion and subroutines with returned capture groups

Since PCRE2 10.46, recursion and subroutine calls may also specify a list of capture groups to return. This is a PCRE2 syntax extension not supported by Perl. The pattern matching recurses into the referenced expression as described above, however, when the recursion returns to the calling expression the subgroups captured during the recursion can be retained when the calling expression's context is restored.

When used as a subroutine, this allows the subroutine's capture groups to be used as return values.

Only the specific capture groups listed by the caller will be retained, using the following syntax:

  (?R(grouplist))       recurse whole pattern, returning capture groups
  (?n(grouplist))       )
  (?+n(grouplist))      )
  (?-n(grouplist))      ) call subroutine, returning capture groups
  (?&name(grouplist))   )
  (?P>name(grouplist))  )

The list of capture groups "grouplist" is a comma-separated list of (absolute or relative) group numbers, and group names enclosed in single quotes or angle brackets.

Here is an example which first uses the DEFINE condition to create a re-usable routine for matching a weekday, then calls that subroutine and retains the groups it captures for use later:

  (?x: # ignore whitespace for clarity
    # Define the routine "weekendday" which matches Saturday or
    # Sunday, and returns the Sat/Sun prefix as \k<short>.
    (?(DEFINE) (?<weekendday>
        (?|(?<short>Sat)urday|(?<short>Sun)day) ) )
    # Call the routine. Matches "Saturday,Sat" or "Sunday,Sun".
    (?&weekendday(<short>)),\k<short> )

This feature is not available using the Oniguruma syntax \g<...> or \g'...' below.

Oniguruma Subroutine Syntax

For compatibility with Oniguruma, the non-Perl syntax \g followed by a name or a number enclosed either in angle brackets or single quotes, is an alternative syntax for calling a group as a subroutine, possibly recursively. Here are two of the examples used above, rewritten using this syntax:

(?<pn> \( (  (?>[^()]+) |  \g<pn> )* \) )
(sens|respons)e and \g'1'ibility

PCRE2 supports an extension to Oniguruma: if a number is preceded by a plus or a minus sign it is taken as a relative reference. For example:

(abc)(?i:\g<-1>)

Note that \g{...} (Perl syntax) and \g<...> (Oniguruma syntax) are not synonymous. The former is a backreference; the latter is a subroutine call.

Backtracking Control

There are a number of special "Backtracking Control Verbs" (to use Perl's terminology) that modify the behaviour of backtracking during matching. They are generally of the form (*VERB) or (*VERB:NAME). Some verbs take either form, and may behave differently depending on whether or not a name argument is present. The names are not required to be unique within the pattern.

By default, for compatibility with Perl, a name is any sequence of characters that does not include a closing parenthesis. The name is not processed in any way, and it is not possible to include a closing parenthesis in the name.

A closing parenthesis can be included in a name either as \) or between \Q and \E. In addition to backslash processing, if the extended or PCRE2_EXTENDED_MORE option is also set, unescaped white space in verb names is skipped, and #-comments are recognized, exactly as in the rest of the pattern. extended and PCRE2_EXTENDED_MORE do not affect verb names.

The maximum length of a name is 255 in the 8-bit library. If the name is empty, that is, if the closing parenthesis immediately follows the colon, the effect is as if the colon were not there. Any number of these verbs may occur in a pattern. Except for (*ACCEPT), they may not be quantified.

Since these verbs are specifically related to backtracking, most of them can be used only when the pattern is to be matched using the traditional matching function, because it uses backtracking algorithms. With the exception of (*FAIL), which behaves like a failing negative assertion.

The behaviour of these verbs in repeated groups, assertions, and in capture groups called as subroutines (whether or not recursively) is documented below.

Optimizations that affect backtracking verbs

PCRE2 contains some optimizations that are used to speed up matching by running some checks at the start of each match attempt. For example, it may know the minimum length of matching subject, or that a particular character must be present. When one of these optimizations bypasses the running of a match, any included backtracking verbs will not, of course, be processed. You can suppress the start-of-match optimizations by setting the no_start_optimize option when calling compile/2 or by starting the pattern with (*NO_START_OPT).

Experiments with Perl suggest that it too has similar optimizations, and like PCRE2, turning them off can change the result of a match.

Verbs that act immediately

The following verbs act as soon as they are encountered.

(*ACCEPT) or (*ACCEPT:NAME)

This verb causes the match to end successfully, skipping the remainder of the pattern. However, when it is inside a capture group that is called as a subroutine, only that group is ended successfully. Matching then continues at the outer level. If (*ACCEPT) in triggered in a positive assertion, the assertion succeeds; in a negative assertion, the assertion fails.

If (*ACCEPT) is inside capturing parentheses, the data so far is captured. For example:

A((?:A|B(*ACCEPT)|C)D)

This matches "AB", "AAD", or "ACD"; when it matches "AB", "B" is captured by the outer parentheses.

(*ACCEPT) is the only backtracking verb that is allowed to be quantified because an ungreedy quantification with a minimum of zero acts only when a backtrack happens. Consider, for example,

(A(*ACCEPT)??B)C

where A, B, and C may be complex expressions. After matching "A", the matcher processes "BC"; if that fails, causing a backtrack, (*ACCEPT) is triggered and the match succeeds. In both cases, all but C is captured. Whereas (*COMMIT) (see below) means "fail on backtrack", a repeated (*ACCEPT) of this type means "succeed on backtrack".

Warning

(*ACCEPT) should not be used within a script run group, because it causes an immediate exit from the group, bypassing the script run checking.

(*FAIL) or (*FAIL:NAME)

This verb causes a matching failure, forcing backtracking to occur. It may be abbreviated to (*F). It is equivalent to (?!) but easier to read. The Perl documentation notes that it is probably useful only when combined with (?{}) or (??{}). Those are, of course, Perl features that are not present in PCRE2. The nearest equivalent is the callout feature, as for example in this pattern:

a+(?C)(*FAIL)

A match with the string "aaaa" always fails, but the callout is taken before each backtrack happens (in this example, 10 times).

(*ACCEPT:NAME) and (*FAIL:NAME) behave the same as (*MARK:NAME)(*ACCEPT) and (*MARK:NAME)(*FAIL), respectively, that is, a (*MARK) is recorded just before the verb acts.

Recording which path was taken

There is one verb whose main purpose is to track how a match was arrived at, though it also has a secondary use in conjunction with advancing the match starting point (see (*SKIP) below).

(*MARK:NAME) or (*:NAME)

A name is always required with this verb. For all the other backtracking control verbs, a NAME argument is optional.

When a match succeeds, the name of the last-encountered mark name on the matching path is passed back to the caller as described in the section entitled "Other information about the match" in the pcre2api documentation. This applies to all instances of (*MARK) and other verbs, including those inside assertions and atomic groups. However, there are differences in those cases when (*MARK) is used in conjunction with (*SKIP) as described below.

The mark name that was last encountered on the matching path is passed back. A verb without a NAME argument is ignored for this purpose. Here is an example of pcre2test output, where the "mark" modifier requests the retrieval and outputting of (*MARK) data:

    re> /X(*MARK:A)Y|X(*MARK:B)Z/mark
data> XY
   0: XY
MK: A
XZ
   0: XZ
MK: B

The (*MARK) name is tagged with "MK:" in this output, and in this example it indicates which of the two alternatives matched. This is a more efficient way of obtaining this information than putting each alternative in its own capturing parentheses.

If a verb with a name is encountered in a positive assertion that is true, the name is recorded and passed back if it is the last-encountered. This does not happen for negative assertions or failing positive assertions.

After a partial match or a failed match, the last encountered name in the entire match process is returned. For example:

    re> /X(*MARK:A)Y|X(*MARK:B)Z/mark
data> XP
No match, mark `=`  B

Note that in this unanchored example the mark is retained from the match attempt that started at the letter "X" in the subject. Subsequent match attempts starting at "P" and then with an empty string do not get as far as the (*MARK) item, but nevertheless do not reset it.

If you are interested in (*MARK) values after failed matches, you should probably either set the no_start_optimize option to ensure that the match is always attempted.

Verbs that act after backtracking

The following verbs do nothing when they are encountered. Matching continues with what follows, but if there is a subsequent match failure, causing a backtrack to the verb, a failure is forced. That is, backtracking cannot pass to the left of the verb. However, when one of these verbs appears inside an atomic group or in an atomic lookaround assertion that is true, its effect is confined to that group, because once the group has been matched, there is never any backtracking into it. Backtracking from beyond an atomic assertion or group ignores the entire group, and seeks a preceding backtracking point.

These verbs differ in exactly what kind of failure occurs when backtracking reaches them. The behaviour described below is what happens when the verb is not in a subroutine or an assertion. Subsequent sections cover these special cases.

(*COMMIT) or (*COMMIT:NAME)

This verb causes the whole match to fail outright if there is a later matching failure that causes backtracking to reach it. Even if the pattern is unanchored, no further attempts to find a match by advancing the starting point take place. If (*COMMIT) is the only backtracking verb that is encountered, once it has been passed run/3 is committed to finding a match at the current starting point, or not at all. For example:

a+(*COMMIT)b

This matches "xxaab" but not "aacaab". It can be thought of as a kind of dynamic anchor, or "I've started, so I must finish."

The behaviour of (*COMMIT:NAME) is not the same as (*MARK:NAME)(*COMMIT). It is like (*MARK:NAME) in that the name is remembered for passing back to the caller. However, (*SKIP:NAME) searches only for names that are set with (*MARK), ignoring those set by any of the other backtracking verbs.

If there is more than one backtracking verb in a pattern, a different one that follows (*COMMIT) may be triggered first, so merely passing (*COMMIT) during a match does not always guarantee that a match must be at this starting point.

Note that (*COMMIT) at the start of a pattern is not the same as an anchor, unless PCRE2's start-of-match optimizations are turned off, as shown in this output from pcre2test:

    re> /(*COMMIT)abc/
data> xyzabc
   0: abc
data>
re> /(*COMMIT)abc/no_start_optimize
data> xyzabc
No match

For the first pattern, PCRE2 knows that any match must start with "a", so the optimization skips along the subject to "a" before applying the pattern to the first set of data. The match attempt then succeeds. The second pattern disables the optimization that skips along to the first character. The pattern is now applied starting at "x", and so the (*COMMIT) causes the match to fail without trying any other starting points.

(*PRUNE) or (*PRUNE:NAME)

This verb causes the match to fail at the current starting position in the subject if there is a later matching failure that causes backtracking to reach it. If the pattern is unanchored, the normal "bumpalong" advance to the next starting character then happens. Backtracking can occur as usual to the left of (*PRUNE), before it is reached, or when matching to the right of (*PRUNE), but if there is no match to the right, backtracking cannot cross (*PRUNE). In simple cases, the use of (*PRUNE) is just an alternative to an atomic group or possessive quantifier, but there are some uses of (*PRUNE) that cannot be expressed in any other way. In an anchored pattern (*PRUNE) has the same effect as (*COMMIT).

The behaviour of (*PRUNE:NAME) is not the same as (*MARK:NAME)(*PRUNE). It is like (*MARK:NAME) in that the name is remembered for passing back to the caller. However, (*SKIP:NAME) searches only for names set with (*MARK), ignoring those set by other backtracking verbs.

(*SKIP)

This verb, when given without a name, is like (*PRUNE), except that if the pattern is unanchored, the "bumpalong" advance is not to the next character, but to the position in the subject where (*SKIP) was encountered. (*SKIP) signifies that whatever text was matched leading up to it cannot be part of a successful match if there is a later mismatch. Consider:

a+(*SKIP)b

If the subject is "aaaac...", after the first match attempt fails (starting at the first character in the string), the starting point skips on to start the next attempt at "c". Note that a possessive quantifier does not have the same effect as this example; although it would suppress backtracking during the first match attempt, the second attempt would start at the second character instead of skipping on to "c".

If (*SKIP) is used to specify a new starting position that is the same as the starting position of the current match, or (by being inside a lookbehind) earlier, the position specified by (*SKIP) is ignored, and instead the normal "bumpalong" occurs.

(*SKIP:NAME)

When (*SKIP) has an associated name, its behaviour is modified. When such a (*SKIP) is triggered, the previous path through the pattern is searched for the most recent (*MARK) that has the same name. If one is found, the "bumpalong" advance is to the subject position that corresponds to that (*MARK) instead of to where (*SKIP) was encountered. If no (*MARK) with a matching name is found, the (*SKIP) is ignored.

The search for a (*MARK) name uses the normal backtracking mechanism, which means that it does not see (*MARK) settings that are inside atomic groups or assertions, because they are never re-entered by backtracking. Compare the following pcre2test examples:

    re> /a(?>(*MARK:X))(*SKIP:X)(*F)|(.)/
data: abc
   0: a
   1: a
data:
    re> /a(?:(*MARK:X))(*SKIP:X)(*F)|(.)/
data: abc
   0: b
   1: b

In the first example, the (*MARK) setting is in an atomic group, so it is not seen when (*SKIP:X) triggers, causing the (*SKIP) to be ignored. This allows the second branch of the pattern to be tried at the first character position. In the second example, the (*MARK) setting is not in an atomic group. This allows (*SKIP:X) to find the (*MARK) when it backtracks, and this causes a new matching attempt to start at the second character. This time, the (*MARK) is never seen because "a" does not match "b", so the matcher immediately jumps to the second branch of the pattern.

Note that (*SKIP:NAME) searches only for names set by (*MARK:NAME). It ignores names that are set by other backtracking verbs.

(*THEN) or (*THEN:NAME)

This verb causes a skip to the next innermost alternative when backtracking reaches it. That is, it cancels any further backtracking within the current alternative. Its name comes from the observation that it can be used for a pattern-based if-then-else block:

( COND1 (*THEN) FOO |  COND2 (*THEN) BAR |  COND3 (*THEN) BAZ )  ...

If the COND1 pattern matches, FOO is tried (and possibly further items after the end of the group if FOO succeeds); on failure, the matcher skips to the second alternative and tries COND2, without backtracking into COND1. If that succeeds and BAR fails, COND3 is tried. If subsequently BAZ fails, there are no more alternatives, so there is a backtrack to whatever came before the entire group. If (*THEN) is not inside an alternation, it acts like (*PRUNE).

The behaviour of (*THEN:NAME) is not the same as (*MARK:NAME)(*THEN). It is like (*MARK:NAME) in that the name is remembered for passing back to the caller. However, (*SKIP:NAME) searches only for names set with (*MARK), ignoring those set by other backtracking verbs.

A group that does not contain a | character is just a part of the enclosing alternative; it is not a nested alternation with only one alternative. The effect of (*THEN) extends beyond such a group to the enclosing alternative. Consider this pattern, where A, B, etc. are complex pattern fragments that do not contain any | characters at this level:

A (B(*THEN)C) |  D

If A and B are matched, but there is a failure in C, matching does not backtrack into A; instead it moves to the next alternative, that is, D. However, if the group containing (*THEN) is given an alternative, it behaves differently:

A (B(*THEN)C |  (*FAIL)) |  D

The effect of (*THEN) is now confined to the inner group. After a failure in C, matching moves to (*FAIL), which causes the whole group to fail because there are no more alternatives to try. In this case, matching does backtrack into A.

Note that a conditional group is not considered as having two alternatives, because only one is ever used. In other words, the | character in a conditional group has a different meaning. Ignoring white space, consider:

^.*? (?(?=a) a |  b(*THEN)c )

If the subject is "ba", this pattern does not match. Because .*? is ungreedy, it initially matches zero characters. The condition (?=a) then fails, the character "b" is matched, but "c" is not. At this point, matching does not backtrack to .? as might perhaps be expected from the presence of the | character. The conditional group is part of the single alternative that comprises the whole pattern, and so the match fails. (If there was a backtrack into `.?`, allowing it to match "b", the match would succeed.)

The verbs just described provide four different "strengths" of control when subsequent matching fails. (*THEN) is the weakest, carrying on the match at the next alternative. (*PRUNE) comes next, failing the match at the current starting position, but allowing an advance to the next character (for an unanchored pattern). (*SKIP) is similar, except that the advance may be more than one character. (*COMMIT) is the strongest, causing the entire match to fail.

More than one backtracking verb

If more than one backtracking verb is present in a pattern, the one that is backtracked onto first acts. For example, consider this pattern, where A, B, etc. are complex pattern fragments:

(A(*COMMIT)B(*THEN)C|ABD)

If A matches but B fails, the backtrack to (*COMMIT) causes the entire match to fail. However, if A and B match, but C fails, the backtrack to (*THEN) causes the next alternative (ABD) to be tried. This behaviour is consistent, but is not always the same as Perl's. It means that if two or more backtracking verbs appear in succession, all but the last of them has no effect. Consider this example:

...(*COMMIT)(*PRUNE)...

If there is a matching failure to the right, backtracking onto (*PRUNE) causes it to be triggered, and its action is taken. There can never be a backtrack onto (*COMMIT).

Backtracking verbs in repeated groups

PCRE2 sometimes differs from Perl in its handling of backtracking verbs in repeated groups. For example, consider:

/(a(*COMMIT)b)+ac/

If the subject is "abac", Perl matches unless its optimizations are disabled, but PCRE2 always fails because the (*COMMIT) in the second repeat of the group acts.

Backtracking verbs in assertions

(*FAIL) in any assertion has its normal effect: it forces an immediate backtrack. The behaviour of the other backtracking verbs depends on whether or not the assertion is standalone or acting as the condition in a conditional group.

(*ACCEPT) in a standalone positive assertion causes the assertion to succeed without any further processing; captured strings and a mark name (if set) are retained. In a standalone negative assertion, (*ACCEPT) causes the assertion to fail without any further processing; captured substrings and any mark name are discarded.

If the assertion is a condition, (*ACCEPT) causes the condition to be true for a positive assertion and false for a negative one; captured substrings are retained in both cases.

The remaining verbs act only when a later failure causes a backtrack to reach them. This means that, for the Perl-compatible assertions, their effect is confined to the assertion, because Perl lookaround assertions are atomic. A backtrack that occurs after such an assertion is complete does not jump back into the assertion. Note in particular that a (*MARK) name that is set in an assertion is not "seen" by an instance of (*SKIP:NAME) later in the pattern.

PCRE2 now supports non-atomic positive assertions and also "scan substring" assertions, as described in the sections entitled "Non-atomic assertions" and "Scan substring assertions" above. These assertions must be standalone (not used as conditions). They are not Perl-compatible. For these assertions, a later backtrack does jump back into the assertion, and therefore verbs such as (*COMMIT) can be triggered by backtracks from later in the pattern.

The effect of (*THEN) is not allowed to escape beyond an assertion. If there are no more branches to try, (*THEN) causes a positive assertion to be false, and a negative assertion to be true. This behaviour differs from Perl when the assertion has only one branch.

The other backtracking verbs are not treated specially if they appear in a standalone positive assertion. In a conditional positive assertion, backtracking (from within the assertion) into (*COMMIT), (*SKIP), or (*PRUNE) causes the condition to be false. However, for both standalone and conditional negative assertions, backtracking into (*COMMIT), (*SKIP), or (*PRUNE) causes the assertion to be true, without considering any further alternative branches.

Backtracking verbs in subroutines

These behaviours occur whether or not the group is called recursively.

(*ACCEPT) in a group called as a subroutine causes the subroutine match to succeed without any further processing. Matching then continues after the subroutine call. Perl documents this behaviour. Perl's treatment of the other verbs in subroutines is different in some cases.

(*FAIL) in a group called as a subroutine has its normal effect: it forces an immediate backtrack.

(*COMMIT), (*SKIP), and (*PRUNE) cause the subroutine match to fail when triggered by being backtracked to in a group called as a subroutine. There is then a backtrack at the outer level.

(*THEN), when triggered, skips to the next alternative in the innermost enclosing group that has alternatives (its normal behaviour). However, if there is no such group within the subroutine's group, the subroutine match fails and there is a backtrack at the outer level.

Summary

Types

capture()

compile_option()

compile_options()

exported()

Opaque data type containing an exported compiled regular expression.

mp()

Opaque data type containing a compiled regular expression.

nl_spec()

option()

options()

replace_fun()

Functions

compile(Regexp)

The same as compile(Regexp,[])

compile(Regexp, Options)

Compiles a regular expression, with the syntax described below, into an internal format to be used later as a parameter to run/2 and run/3.

import(Exported)

Imports a regular expression compiled with option export. The importing is cheap if the importing node is compatible enough to the exporting node. If incompatible, as a fallback the import/1 function will re-compile the regular expression from its string format, which is included in exported().

inspect(MP, Item)

Takes a compiled regular expression and an item, and returns the relevant data from the regular expression.

replace(Subject, RE, Replacement)

Equivalent to replace(Subject, RE, Replacement, []).

replace(Subject, RE, Replacement, Options)

Replaces the matched part of the Subject string with Replacement.

run(Subject, RE)

Equivalent to run(Subject, RE, []).

run(Subject, RE, Options)

Executes a regular expression matching, and returns match/{match, Captured} or nomatch.

split(Subject, RE)

Equivalent to split(Subject, RE, []).

split(Subject, RE, Options)

Splits the input into parts by finding tokens according to the regular expression supplied.

version()

The return of this function is a string with the PCRE version of the system that was used in the Erlang/OTP compilation.

Types

capture()

(not exported)

-type capture() ::
          all | all_but_first | all_names | first | none |
          (ValueList :: [integer() | string() | atom()]).

compile_option()

(not exported)

-type compile_option() ::
          unicode | anchored | caseless | dollar_endonly | dotall | extended | firstline | multiline |
          no_auto_capture | dupnames | ungreedy |
          {newline, nl_spec()} |
          bsr_anycrlf | bsr_unicode | no_start_optimize | ucp | never_utf.

compile_options()

-type compile_options() :: [compile_option()].

exported()

(not exported)

-type exported() :: {re_exported_pattern, _, _, _, _}.

Opaque data type containing an exported compiled regular expression.

Type exported() is guaranteed to be a tuple having the atom re_exported_pattern as its first element to allow for matching in guards. The arity of the tuple or the content of the other fields can change in future Erlang/OTP releases.

mp()

-type mp() :: {re_pattern, _, _, _, _}.

Opaque data type containing a compiled regular expression.

A compiled regular expression of this type can only be executed on the node instance where it was compiled.

Change

Before Erlang/OTP 28, it was possible to abuse mp() and execute such terms on nodes other than the one where they were compiled. This was unsafe, as it only worked if the two nodes were sufficiently compatible (hardware and software), and executing incompatible mp() terms could lead to unpleasant side effects.

From Erlang/OTP 28 this unsafe abuse is no longer possible. Instead use the possibility to safely export and import compiled regular expressions between nodes.

Type mp() is guaranteed to be a tuple having the atom re_pattern as its first element to allow for matching in guards. The arity of the tuple or the content of the other fields can change in future Erlang/OTP releases.

nl_spec()

(not exported)

-type nl_spec() :: cr | crlf | lf | nul | anycrlf | any.

option()

(not exported)

-type option() ::
          anchored | global | notbol | noteol | notempty | notempty_atstart | report_errors |
          {offset, non_neg_integer()} |
          {match_limit, non_neg_integer()} |
          {match_limit_recursion, non_neg_integer()} |
          {capture, ValueSpec :: capture()} |
          {capture, ValueSpec :: capture(), Type :: index | list | binary} |
          compile_option().

options()

-type options() :: [option()].

replace_fun()

(not exported)

-type replace_fun() :: fun((binary(), [binary()]) -> iodata() | unicode:charlist()).

Functions

compile(Regexp)

-spec compile(Regexp) -> {ok, mp()} | {error, ErrSpec}
                 when
                     Regexp :: iodata(),
                     ErrSpec :: {ErrString :: string(), Position :: non_neg_integer()}.

The same as compile(Regexp,[])

compile(Regexp, Options)

-spec compile(Regexp, Options) -> {ok, mp() | exported()} | {error, ErrSpec}
                 when
                     Regexp :: iodata() | unicode:charlist(),
                     Options :: [Option],
                     Option :: compile_option() | export,
                     ErrSpec :: {ErrString :: string(), Position :: non_neg_integer()}.

Compiles a regular expression, with the syntax described below, into an internal format to be used later as a parameter to run/2 and run/3.

Compiling the regular expression before matching is useful if the same expression is to be used in matching against multiple subjects during the lifetime of the program. Compiling once and executing many times is far more efficient than compiling each time one wants to match.

When option unicode is specified, the regular expression is to be specified as a valid Unicode charlist(), otherwise as any valid iodata/0.

Options:

unicode - The regular expression is specified as a Unicode charlist() and the resulting regular expression code is to be run against a valid Unicode charlist() subject. Also consider option ucp when using Unicode characters.

anchored - The pattern is forced to be "anchored", that is, it is constrained to match only at the first matching point in the string that is searched (the "subject string"). This effect can also be achieved by appropriate constructs in the pattern itself.

caseless - Letters in the pattern match both uppercase and lowercase letters. It is equivalent to Perl option /i and can be changed within a pattern by a (?i) option setting. Uppercase and lowercase letters are defined as in the ISO 8859-1 character set.

dollar_endonly - A dollar metacharacter in the pattern matches only at the end of the subject string. Without this option, a dollar also matches immediately before a newline at the end of the string (but not before any other newlines). This option is ignored if option multiline is specified. There is no equivalent option in Perl, and it cannot be set within a pattern.

dotall - A dot in the pattern matches all characters, including those indicating newline. Without it, a dot does not match when the current position is at a newline. This option is equivalent to Perl option /s and it can be changed within a pattern by a (?s) option setting. A negative class, such as [^a], always matches newline characters, independent of the setting of this option.

extended - If this option is set, most white space characters in the pattern are totally ignored except when escaped or inside a character class. However, white space is not allowed within sequences such as (?> that introduce various parenthesized subpatterns, nor within a numerical quantifier such as {1,3}. However, ignorable white space is permitted between an item and a following quantifier and between a quantifier and a following + that indicates possessiveness.

White space did not used to include the VT character (code 11), because Perl did not treat this character as white space. However, Perl changed at release 5.18, so PCRE followed at release 8.34, and VT is now treated as white space.

This also causes characters between an unescaped # outside a character class and the next newline, inclusive, to be ignored. This is equivalent to Perl's /x option, and it can be changed within a pattern by a (?x) option setting.

With this option, comments inside complicated patterns can be included. However, notice that this applies only to data characters. Whitespace characters can never appear within special character sequences in a pattern, for example within sequence (?( that introduces a conditional subpattern.

firstline - An unanchored pattern is required to match before or at the first newline in the subject string, although the matched text can continue over the newline.

multiline - By default, PCRE treats the subject string as consisting of a single line of characters (even if it contains newlines). The "start of line" metacharacter (^) matches only at the start of the string, while the "end of line" metacharacter ($) matches only at the end of the string, or before a terminating newline (unless option dollar_endonly is specified). This is the same as in Perl.

When this option is specified, the "start of line" and "end of line" constructs match immediately following or immediately before internal newlines in the subject string, respectively, as well as at the very start and end. This is equivalent to Perl option /m and can be changed within a pattern by a (?m) option setting. If there are no newlines in a subject string, or no occurrences of ^ or $ in a pattern, setting multiline has no effect.

no_auto_capture - Disables the use of numbered capturing parentheses in the pattern. Any opening parenthesis that is not followed by ? behaves as if it is followed by ?:. Named parentheses can still be used for capturing (and they acquire numbers in the usual way). There is no equivalent option in Perl.

dupnames - Names used to identify capturing subpatterns need not be unique. This can be helpful for certain types of pattern when it is known that only one instance of the named subpattern can ever be matched. More details of named subpatterns are provided below.

ungreedy - Inverts the "greediness" of the quantifiers so that they are not greedy by default, but become greedy if followed by "?". It is not compatible with Perl. It can also be set by a (?U) option setting within the pattern.

{newline, NLSpec} - Overrides the default definition of a newline in the subject string, which is LF (ASCII 10) in Erlang.

cr - Newline is indicated by a single character cr (ASCII 13).

lf - Newline is indicated by a single character LF (ASCII 10), the default.

nul - Newline is indicated by a single character NUL (ASCII 0).

crlf - Newline is indicated by the two-character CRLF (ASCII 13 followed by ASCII 10) sequence.

anycrlf - Any of the three preceding sequences is to be recognized.

any - Any of the newline sequences above, and the Unicode sequences VT (vertical tab, U+000B), FF (formfeed, U+000C), NEL (next line, U+0085), LS (line separator, U+2028), and PS (paragraph separator, U+2029).

bsr_anycrlf - Specifies specifically that \R is to match only the CR, LF, or CRLF sequences, not the Unicode-specific newline characters.

bsr_unicode - Specifies specifically that \R is to match all the Unicode newline characters (including CRLF, and so on, the default).

no_start_optimize - Disables optimization that can malfunction if "Special start-of-pattern items" are present in the regular expression. A typical example would be when matching "DEFABC" against "(COMMIT)ABC", where the start optimization of PCRE would skip the subject up to "A" and never realize that the (COMMIT) instruction is to have made the matching fail. This option is only relevant if you use "start-of-pattern items", as discussed in section PCRE Regular Expression Details.

ucp - Specifies that Unicode character properties are to be used when resolving \B, \b, \D, \d, \S, \s, \W and \w. Without this flag, only ISO Latin-1 properties are used. Using Unicode properties hurts performance, but is semantically correct when working with Unicode characters beyond the ISO Latin-1 range.

never_utf - Specifies that the (UTF) and/or (UTF8) "start-of-pattern items" are forbidden. This flag cannot be combined with option unicode. Useful if ISO Latin-1 patterns from an external source are to be compiled.

export - Returns the compiled regular expression in a format exported() that can be imported into any node instance. Normally, a compiled regex can only be executed by the node instance that compiled it. With option export the returned term can be communicated (in any way) to any node. The receiving node must import it by calling import/1 to get a compiled regular expression executable on that local node.

Option export and function import/1 are supported since OTP 28.1.

import(Exported)

(since OTP 28.1)

-spec import(Exported :: exported()) -> mp().

Imports a regular expression compiled with option export. The importing is cheap if the importing node is compatible enough to the exporting node. If incompatible, as a fallback the import/1 function will re-compile the regular expression from its string format, which is included in exported().

inspect(MP, Item)

(since OTP 17.0)

-spec inspect(MP, Item) -> {namelist, [binary()]} when MP :: mp(), Item :: namelist.

Takes a compiled regular expression and an item, and returns the relevant data from the regular expression.

The only supported item is namelist, which returns the tuple {namelist, [binary()]}, containing the names of all (unique) named subpatterns in the regular expression.

For example:

1> {ok,MP} = re:compile("(?<A>A)|(?<B>B)|(?<C>C)").
{ok,{re_pattern,3,0,0,
                <<69,82,67,80,119,0,0,0,0,0,0,0,1,0,0,0,255,255,255,255,
                  255,255,...>>}}
2> re:inspect(MP,namelist).
{namelist,[<<"A">>,<<"B">>,<<"C">>]}
3> {ok,MPD} = re:compile("(?<C>A)|(?<B>B)|(?<C>C)",[dupnames]).
{ok,{re_pattern,3,0,0,
                <<69,82,67,80,119,0,0,0,0,0,8,0,1,0,0,0,255,255,255,255,
                  255,255,...>>}}
4> re:inspect(MPD,namelist).
{namelist,[<<"B">>,<<"C">>]}

Notice in the second example that the duplicate name only occurs once in the returned list, and that the list is in alphabetical order regardless of where the names are positioned in the regular expression. The order of the names is the same as the order of captured subexpressions if {capture, all_names} is specified as an option to run/3. You can therefore create a name-to-value mapping from the result of run/3 like this:

1> {ok,MP} = re:compile("(?<A>A)|(?<B>B)|(?<C>C)").
{ok,{re_pattern,3,0,0,
                <<69,82,67,80,119,0,0,0,0,0,0,0,1,0,0,0,255,255,255,255,
                  255,255,...>>}}
2> {namelist, N} = re:inspect(MP,namelist).
{namelist,[<<"A">>,<<"B">>,<<"C">>]}
3> {match,L} = re:run("AA",MP,[{capture,all_names,binary}]).
{match,[<<"A">>,<<>>,<<>>]}
4> NameMap = lists:zip(N,L).
[{<<"A">>,<<"A">>},{<<"B">>,<<>>},{<<"C">>,<<>>}]

replace(Subject, RE, Replacement)

-spec replace(Subject, RE, Replacement) -> iodata() | unicode:charlist()
                 when
                     Subject :: iodata() | unicode:charlist(),
                     RE :: mp() | iodata(),
                     Replacement :: iodata() | unicode:charlist() | replace_fun().

Equivalent to replace(Subject, RE, Replacement, []).

replace(Subject, RE, Replacement, Options)

-spec replace(Subject, RE, Replacement, Options) -> iodata() | unicode:charlist()
                 when
                     Subject :: iodata() | unicode:charlist(),
                     RE :: mp() | iodata() | unicode:charlist(),
                     Replacement :: iodata() | unicode:charlist() | replace_fun(),
                     Options :: [Option],
                     Option ::
                         anchored | global | notbol | noteol | notempty | notempty_atstart |
                         {offset, non_neg_integer()} |
                         {match_limit, non_neg_integer()} |
                         {match_limit_recursion, non_neg_integer()} |
                         {return, ReturnType} |
                         CompileOpt,
                     ReturnType :: iodata | list | binary,
                     CompileOpt :: compile_option().

Replaces the matched part of the Subject string with Replacement.

The permissible options are the same as for run/3, except that optioncapture is not allowed. Instead a {return, ReturnType} is present. The default return type is iodata, constructed in a way to minimize copying. The iodata result can be used directly in many I/O operations. If a flat list/0 is desired, specify {return, list}. If a binary is desired, specify {return, binary}.

As in function run/3, an mp/0 compiled with option unicode requires Subject to be a Unicode charlist(). If compilation is done implicitly and the unicode compilation option is specified to this function, both the regular expression and Subject are to specified as valid Unicode charlist()s.

If the replacement is given as a string, it can contain the special character &, which inserts the whole matching expression in the result, and the special sequence \N (where N is an integer > 0), \gN, or \g{N}, resulting in the subexpression number N, is inserted in the result. If no subexpression with that number is generated by the regular expression, nothing is inserted.

To insert an & or a \ in the result, precede it with a \. Notice that Erlang already gives a special meaning to \ in literal strings, so a single \ must be written as "\\" and therefore a double \ as "\\\\".

Example:

1> re:replace("abcd","c","[&]",[{return,list}]).
"ab[c]d"

while

2> re:replace("abcd","c","[\\&]",[{return,list}]).
"ab[&]d"

If the replacement is given as a fun, it will be called with the whole matching expression as the first argument and a list of subexpression matches in the order in which they appear in the regular expression. The returned value will be inserted in the result.

Example:

3> re:replace("abcd", ".(.)",
    fun(Whole, [<<C>>]) ->
         <<$#, Whole/binary, $-, (C - $a + $A), $#>>
    end,
    [{return, list}]).
"#ab-B#cd"

Note

Non-matching optional subexpressions will not be included in the list of subexpression matches if they are the last subexpressions in the regular expression.

Example:

The regular expression "(a)(b)?(c)?" ("a", optionally followed by "b", optionally followed by "c") will create the following subexpression lists:

[<<"a">>, <<"b">>, <<"c">>] when applied to the string "abc"

[<<"a">>, <<>>, <<"c">>] when applied to the string "acx"

[<<"a">>, <<"b">>] when applied to the string "abx"

[<<"a">>] when applied to the string "axx"

As with run/3, compilation errors raise the badarg exception. compile/2 can be used to get more information about the error.

run(Subject, RE)

-spec run(Subject, RE) -> {match, Captured} | nomatch
             when
                 Subject :: iodata() | unicode:charlist(),
                 RE :: mp() | iodata(),
                 Captured :: [CaptureData],
                 CaptureData :: {integer(), integer()}.

Equivalent to run(Subject, RE, []).

run(Subject, RE, Options)

-spec run(Subject, RE, Options) -> {match, Captured} | match | nomatch | {error, ErrType}
             when
                 Subject :: iodata() | unicode:charlist(),
                 RE :: mp() | iodata() | unicode:charlist(),
                 Options :: options(),
                 Captured :: [CaptureData] | [[CaptureData]],
                 CaptureData :: {integer(), integer()} | ListConversionData | binary(),
                 ListConversionData ::
                     string() | {error, string(), binary()} | {incomplete, string(), binary()},
                 ErrType :: match_limit | match_limit_recursion | {compile, CompileErr},
                 CompileErr :: {ErrString :: string(), Position :: non_neg_integer()}.

Executes a regular expression matching, and returns match/{match, Captured} or nomatch.

The regular expression can be specified either as iodata/0 in which case it is automatically compiled (as by compile/2) and executed, or as a precompiled mp() in which case it is executed against the subject directly.

When compilation is involved, exception badarg is thrown if a compilation error occurs. Call compile/2 to get information about the location of the error in the regular expression.

If the regular expression is previously compiled, the option list can only contain the following options:

anchored

{capture, ValueSpec}/{capture, ValueSpec, Type}

global

{match_limit, integer() >= 0}

{match_limit_recursion, integer() >= 0}

notbol

notempty

notempty_atstart

noteol

{offset, integer() >= 0}

report_errors

Otherwise all options valid for function compile/2 are also allowed. Options allowed both for compilation and execution of a match, namely anchored, affect both the compilation and execution if present together with a non-precompiled regular expression.

Change

As from Erlang/OTP 28, options {newline, _}, bsr_anycrlf and bsr_unicode can only be used to control the compilation of a regular expression. They will no longer be accepted by run/3, replace/4 and split/3 if the regular expression was previously compiled and the options do not comply with what was given at compile time.

If the regular expression was previously compiled with option unicode, Subject is to be provided as a valid Unicode charlist(), otherwise any iodata/0 will do. If compilation is involved and option unicode is specified, both Subject and the regular expression are to be specified as valid Unicode charlists().

{capture, ValueSpec}/{capture, ValueSpec, Type} defines what to return from the function upon successful matching. The capture tuple can contain both a value specification, telling which of the captured substrings are to be returned, and a type specification, telling how captured substrings are to be returned (as index tuples, lists, or binaries). The options are described in detail below.

If the capture options describe that no substring capturing is to be done ({capture, none}), the function returns the single atom match upon successful matching, otherwise the tuple {match, ValueList}. Disabling capturing can be done either by specifying none or an empty list as ValueSpec.

Option report_errors adds the possibility that an error tuple is returned. The tuple either indicates a matching error (match_limit or match_limit_recursion), or a compilation error, where the error tuple has the format {error, {compile, CompileErr}}. Notice that if option report_errors is not specified, the function never returns error tuples, but reports compilation errors as a badarg exception and failed matches because of exceeded match limits simply as nomatch.

The following options are relevant for execution:

anchored - Limits run/3 to matching at the first matching position. If a pattern was compiled with anchored, or turned out to be anchored by virtue of its contents, it cannot be made unanchored at matching time, hence there is no unanchored option.

global - Implements global (repetitive) search (flag g in Perl). Each match is returned as a separate list/0 containing the specific match and any matching subexpressions (or as specified by option capture. The Captured part of the return value is hence a list/0 of list/0s when this option is specified.

The interaction of option global with a regular expression that matches an empty string surprises some users. When option global is specified, run/3 handles empty matches in the same way as Perl: a zero-length match at any point is also retried with options [anchored, notempty_atstart]. If that search gives a result of length > 0, the result is included. Example:

re:run("cat","(|at)",[global]).

The following matchings are performed:

At offset 0 - The regular expression (|at) first match at the initial position of string cat, giving the result set [{0,0},{0,0}] (the second {0,0} is because of the subexpression marked by the parentheses). As the length of the match is 0, we do not advance to the next position yet.

At offset 0 with [anchored, notempty_atstart] - The search is retried with options [anchored, notempty_atstart] at the same position, which does not give any interesting result of longer length, so the search position is advanced to the next character (a).

At offset 1 - The search results in [{1,0},{1,0}], so this search is also repeated with the extra options.

At offset 1 with [anchored, notempty_atstart] - Alternative ab is found and the result is [{1,2},{1,2}]. The result is added to the list of results and the position in the search string is advanced two steps.

At offset 3 - The search once again matches the empty string, giving [{3,0},{3,0}].

At offset 1 with [anchored, notempty_atstart] - This gives no result of length > 0 and we are at the last position, so the global search is complete.

The result of the call is:

{match,[[{0,0},{0,0}],[{1,0},{1,0}],[{1,2},{1,2}],[{3,0},{3,0}]]}

notempty - An empty string is not considered to be a valid match if this option is specified. If alternatives in the pattern exist, they are tried. If all the alternatives match the empty string, the entire match fails.

Example:

If the following pattern is applied to a string not beginning with "a" or "b", it would normally match the empty string at the start of the subject:

a?b?

With option notempty, this match is invalid, so run/3 searches further into the string for occurrences of "a" or "b".

notempty_atstart - Like notempty, except that an empty string match that is not at the start of the subject is permitted. If the pattern is anchored, such a match can occur only if the pattern contains \K.

Perl has no direct equivalent of notempty or notempty_atstart, but it does make a special case of a pattern match of the empty string within its split() function, and when using modifier /g. The Perl behavior can be emulated after matching a null string by first trying the match again at the same offset with notempty_atstart and anchored, and then, if that fails, by advancing the starting offset (see below) and trying an ordinary match again.

notbol - Specifies that the first character of the subject string is not the beginning of a line, so the circumflex metacharacter is not to match before it. Setting this without multiline (at compile time) causes circumflex never to match. This option only affects the behavior of the circumflex metacharacter. It does not affect \A.

noteol - Specifies that the end of the subject string is not the end of a line, so the dollar metacharacter is not to match it nor (except in multiline mode) a newline immediately before it. Setting this without multiline (at compile time) causes dollar never to match. This option affects only the behavior of the dollar metacharacter. It does not affect \Z or \z.

report_errors - Gives better control of the error handling in run/3. When specified, compilation errors (if the regular expression is not already compiled) and runtime errors are explicitly returned as an error tuple.

The following are the possible runtime errors:

match_limit - The PCRE library sets a limit on how many times the internal match function can be called. Defaults to 10,000,000 in the library compiled for Erlang. If {error, match_limit} is returned, the execution of the regular expression has reached this limit. This is normally to be regarded as a nomatch, which is the default return value when this occurs, but by specifying report_errors, you are informed when the match fails because of too many internal calls.

match_limit_recursion - This error is very similar to match_limit, but occurs when the internal match function of PCRE is "recursively" called more times than the match_limit_recursion limit, which defaults to 10,000,000 as well. Notice that as long as the match_limit and match_limit_default values are kept at the default values, the match_limit_recursion error cannot occur, as the match_limit error occurs before that (each recursive call is also a call, but not conversely). Both limits can however be changed, either by setting limits directly in the regular expression string (see section PCRE Regular Expression Details) or by specifying options to run/3.

It is important to understand that what is referred to as "recursion" when limiting matches is not recursion on the C stack of the Erlang machine or on the Erlang process stack. The PCRE version compiled into the Erlang VM uses machine "heap" memory to store values that must be kept over recursion in regular expression matches.

{match_limit, integer() >= 0} - Limits the execution time of a match in an implementation-specific way. It is described as follows by the PCRE documentation:

The match_limit field provides a means of preventing PCRE from using up a vast amount of resources when running patterns that are not going to match, but which have a very large number of possibilities in their search trees. The classic example is a pattern that uses nested unlimited repeats.

Internally, pcre_exec() uses a function called match(), which it calls repeatedly (sometimes recursively). The limit set by match_limit is imposed on the number of times this function is called during a match, which has the effect of limiting the amount of backtracking that can take place. For patterns that are not anchored, the count restarts from zero for each position in the subject string.

This means that runaway regular expression matches can fail faster if the limit is lowered using this option. The default value 10,000,000 is compiled into the Erlang VM.

Note

This option does in no way affect the execution of the Erlang VM in terms of "long running BIFs". run/3 always gives control back to the scheduler of Erlang processes at intervals that ensures the real-time properties of the Erlang system.

{match_limit_recursion, integer() >= 0} - Limits the execution time and memory consumption of a match in an implementation-specific way, very similar to match_limit. It is described as follows by the PCRE documentation:

The match_limit_recursion field is similar to match_limit, but instead of limiting the total number of times that match() is called, it limits the depth of recursion. The recursion depth is a smaller number than the total number of calls, because not all calls to match() are recursive. This limit is of use only if it is set smaller than match_limit.

Limiting the recursion depth limits the amount of machine stack that can be used, or, when PCRE has been compiled to use memory on the heap instead of the stack, the amount of heap memory that can be used.

The Erlang VM uses a PCRE library where heap memory is used when regular expression match recursion occurs. This therefore limits the use of machine heap, not C stack.

Specifying a lower value can result in matches with deep recursion failing, when they should have matched:

1> re:run("aaaaaaaaaaaaaz","(a+)*z").
{match,[{0,14},{0,13}]}
2> re:run("aaaaaaaaaaaaaz","(a+)*z",[{match_limit_recursion,5}]).
nomatch
3> re:run("aaaaaaaaaaaaaz","(a+)*z",[{match_limit_recursion,5},report_errors]).
{error,match_limit_recursion}

This option and option match_limit are only to be used in rare cases. Understanding of the PCRE library internals is recommended before tampering with these limits.

{offset, integer() >= 0} - Start matching at the offset (position) specified in the subject string. The offset is zero-based, so that the default is {offset,0} (all of the subject string).

{capture, ValueSpec}/{capture, ValueSpec, Type} - Specifies which captured substrings are returned and in what format. By default, run/3 captures all of the matching part of the substring and all capturing subpatterns (all of the pattern is automatically captured). The default return type is (zero-based) indexes of the captured parts of the string, specified as {Offset,Length} pairs (the index Type of capturing).

As an example of the default behavior, the following call returns, as first and only captured string, the matching part of the subject ("abcd" in the middle) as an index pair {3,4}, where character positions are zero-based, just as in offsets:

re:run("ABCabcdABC","abcd",[]).

The return value of this call is:

{match,[{3,4}]}

Another (and quite common) case is where the regular expression matches all of the subject:

re:run("ABCabcdABC",".*abcd.*",[]).

Here the return value correspondingly points out all of the string, beginning at index 0, and it is 10 characters long:

{match,[{0,10}]}

If the regular expression contains capturing subpatterns, like in:

re:run("ABCabcdABC",".*(abcd).*",[]).

all of the matched subject is captured, as well as the captured substrings:

{match,[{0,10},{3,4}]}

The complete matching pattern always gives the first return value in the list and the remaining subpatterns are added in the order they occurred in the regular expression.

The capture tuple is built up as follows:

ValueSpec - Specifies which captured (sub)patterns are to be returned. ValueSpec can either be an atom describing a predefined set of return values, or a list containing the indexes or the names of specific subpatterns to return.

The following are the predefined sets of subpatterns:

all - All captured subpatterns including the complete matching string. This is the default.

all_names - All named subpatterns in the regular expression, as if a list/0 of all the names in alphabetical order was specified. The list of all names can also be retrieved with inspect/2.

first - Only the first captured subpattern, which is always the complete matching part of the subject. All explicitly captured subpatterns are discarded.

all_but_first - All but the first matching subpattern, that is, all explicitly captured subpatterns, but not the complete matching part of the subject string. This is useful if the regular expression as a whole matches a large part of the subject, but the part you are interested in is in an explicitly captured subpattern. If the return type is list or binary, not returning subpatterns you are not interested in is a good way to optimize.

none - Returns no matching subpatterns, gives the single atom match as the return value of the function when matching successfully instead of the {match, list()} return. Specifying an empty list gives the same behavior.

The value list is a list of indexes for the subpatterns to return, where index 0 is for all of the pattern, and 1 is for the first explicit capturing subpattern in the regular expression, and so on. When using named captured subpatterns (see below) in the regular expression, one can use atom/0s or string/0s to specify the subpatterns to be returned. For example, consider the regular expression:

".*(abcd).*"

matched against string "ABCabcdABC", capturing only the "abcd" part (the first explicit subpattern):

re:run("ABCabcdABC",".*(abcd).*",[{capture,[1]}]).

The call gives the following result, as the first explicitly captured subpattern is "(abcd)", matching "abcd" in the subject, at (zero-based) position 3, of length 4:

{match,[{3,4}]}

Consider the same regular expression, but with the subpattern explicitly named 'FOO':

".*(?<FOO>abcd).*"

With this expression, we could still give the index of the subpattern with the following call:

re:run("ABCabcdABC",".*(?<FOO>abcd).*",[{capture,[1]}]).

giving the same result as before. But, as the subpattern is named, we can also specify its name in the value list:

re:run("ABCabcdABC",".*(?<FOO>abcd).*",[{capture,['FOO']}]).

This would give the same result as the earlier examples, namely:

{match,[{3,4}]}

The values list can specify indexes or names not present in the regular expression, in which case the return values vary depending on the type. If the type is index, the tuple {-1,0} is returned for values with no corresponding subpattern in the regular expression, but for the other types (binary and list), the values are the empty binary or list, respectively.

Type - Optionally specifies how captured substrings are to be returned. If omitted, the default of index is used.

Type can be one of the following:

index - Returns captured substrings as pairs of byte indexes into the subject string and length of the matching string in the subject (as if the subject string was flattened with erlang:iolist_to_binary/1 or unicode:characters_to_binary/2 before matching). Notice that option unicode results in byte-oriented indexes in a (possibly virtual) UTF-8 encoded binary. A byte index tuple {0,2} can therefore represent one or two characters when unicode is in effect. This can seem counter-intuitive, but has been deemed the most effective and useful way to do it. To return lists instead can result in simpler code if that is desired. This return type is the default.

list - Returns matching substrings as lists of characters (Erlang string/0s). It option unicode is used in combination with the \C sequence in the regular expression, a captured subpattern can contain bytes that are not valid UTF-8 (\C matches bytes regardless of character encoding). In that case the list capturing can result in the same types of tuples that unicode:characters_to_list/2 can return, namely three-tuples with tag incomplete or error, the successfully converted characters and the invalid UTF-8 tail of the conversion as a binary. The best strategy is to avoid using the \C sequence when capturing lists.

binary - Returns matching substrings as binaries. If option unicode is used, these binaries are in UTF-8. If the \C sequence is used together with unicode, the binaries can be invalid UTF-8.

In general, subpatterns that were not assigned a value in the match are returned as the tuple {-1,0} when type is index. Unassigned subpatterns are returned as the empty binary or list, respectively, for other return types. Consider the following regular expression:

".*((?<FOO>abdd)|a(..d)).*"

There are three explicitly capturing subpatterns, where the opening parenthesis position determines the order in the result, hence ((?<FOO>abdd)|a(..d)) is subpattern index 1, (?<FOO>abdd) is subpattern index 2, and (..d) is subpattern index 3. When matched against the following string:

"ABCabcdABC"

the subpattern at index 2 does not match, as "abdd" is not present in the string, but the complete pattern matches (because of the alternative a(..d)). The subpattern at index 2 is therefore unassigned and the default return value is:

{match,[{0,10},{3,4},{-1,0},{4,3}]}

Setting the capture Type to binary gives:

{match,[<<"ABCabcdABC">>,<<"abcd">>,<<>>,<<"bcd">>]}

Here the empty binary (<<>>) represents the unassigned subpattern. In the binary case, some information about the matching is therefore lost, as <<>> can also be an empty string captured.

If differentiation between empty matches and non-existing subpatterns is necessary, use the type index and do the conversion to the final type in Erlang code.

When option global is speciified, the capture specification affects each match separately, so that:

re:run("cacb","c(a|b)",[global,{capture,[1],list}]).

gives

{match,[["a"],["b"]]}

For a descriptions of options only affecting the compilation step, see compile/2.

split(Subject, RE)

-spec split(Subject, RE) -> SplitList
               when
                   Subject :: iodata() | unicode:charlist(),
                   RE :: mp() | iodata(),
                   SplitList :: [iodata() | unicode:charlist()].

Equivalent to split(Subject, RE, []).

split(Subject, RE, Options)

-spec split(Subject, RE, Options) -> SplitList
               when
                   Subject :: iodata() | unicode:charlist(),
                   RE :: mp() | iodata() | unicode:charlist(),
                   Options :: [Option],
                   Option ::
                       anchored | notbol | noteol | notempty | notempty_atstart |
                       {offset, non_neg_integer()} |
                       {match_limit, non_neg_integer()} |
                       {match_limit_recursion, non_neg_integer()} |
                       {return, ReturnType} |
                       {parts, NumParts} |
                       group | trim | CompileOpt,
                   NumParts :: non_neg_integer() | infinity,
                   ReturnType :: iodata | list | binary,
                   CompileOpt :: compile_option(),
                   SplitList :: [RetData] | [GroupedRetData],
                   GroupedRetData :: [RetData],
                   RetData :: iodata() | unicode:charlist() | binary() | list().

Splits the input into parts by finding tokens according to the regular expression supplied.

The splitting is basically done by running a global regular expression match and dividing the initial string wherever a match occurs. The matching part of the string is removed from the output.

As in run/3, an mp/0 compiled with option unicode requires Subject to be a Unicode charlist(). If compilation is done implicitly and the unicode compilation option is specified to this function, both the regular expression and Subject are to be specified as valid Unicode charlist()s.

The result is given as a list of "strings", the preferred data type specified in option return (default iodata).

If subexpressions are specified in the regular expression, the matching subexpressions are returned in the resulting list as well. For example:

re:split("Erlang","[ln]",[{return,list}]).

gives

["Er","a","g"]

while

re:split("Erlang","([ln])",[{return,list}]).

gives

["Er","l","a","n","g"]

The text matching the subexpression (marked by the parentheses in the regular expression) is inserted in the result list where it was found. This means that concatenating the result of a split where the whole regular expression is a single subexpression (as in the last example) always results in the original string.

As there is no matching subexpression for the last part in the example (the "g"), nothing is inserted after that. To make the group of strings and the parts matching the subexpressions more obvious, one can use option group, which groups together the part of the subject string with the parts matching the subexpressions when the string was split:

re:split("Erlang","([ln])",[{return,list},group]).

gives

[["Er","l"],["a","n"],["g"]]

Here the regular expression first matched the "l", causing "Er" to be the first part in the result. When the regular expression matched, the (only) subexpression was bound to the "l", so the "l" is inserted in the group together with "Er". The next match is of the "n", making "a" the next part to be returned. As the subexpression is bound to substring "n" in this case, the "n" is inserted into this group. The last group consists of the remaining string, as no more matches are found.

By default, all parts of the string, including the empty strings, are returned from the function, for example:

re:split("Erlang","[lg]",[{return,list}]).

gives

["Er","an",[]]

as the matching of the "g" in the end of the string leaves an empty rest, which is also returned. This behavior differs from the default behavior of the split function in Perl, where empty strings at the end are by default removed. To get the "trimming" default behavior of Perl, specify trim as an option:

re:split("Erlang","[lg]",[{return,list},trim]).

gives

["Er","an"]

The "trim" option says; "give me as many parts as possible except the empty ones", which sometimes can be useful. You can also specify how many parts you want, by specifying {parts,N}:

re:split("Erlang","[lg]",[{return,list},{parts,2}]).

gives

["Er","ang"]

Notice that the last part is "ang", not "an", as splitting was specified into two parts, and the splitting stops when enough parts are given, which is why the result differs from that of trim.

More than three parts are not possible with this indata, so

re:split("Erlang","[lg]",[{return,list},{parts,4}]).

gives the same result as the default, which is to be viewed as "an infinite number of parts".

Specifying 0 as the number of parts gives the same effect as option trim. If subexpressions are captured, empty subexpressions matched at the end are also stripped from the result if trim or {parts,0} is specified.

The trim behavior corresponds exactly to the Perl default. {parts,N}, where N is a positive integer, corresponds exactly to the Perl behavior with a positive numerical third parameter. The default behavior of split/3 corresponds to the Perl behavior when a negative integer is specified as the third parameter for the Perl routine.

Summary of options not previously described for function run/3:

{return,ReturnType} - Specifies how the parts of the original string are presented in the result list. Valid types:

iodata - The variant of iodata/0 that gives the least copying of data with the current implementation (often a binary, but do not depend on it).

binary - All parts returned as binaries.

list - All parts returned as lists of characters ("strings").

group - Groups together the part of the string with the parts of the string matching the subexpressions of the regular expression.

The return value from the function is in this case a list/0 of list/0s. Each sublist begins with the string picked out of the subject string, followed by the parts matching each of the subexpressions in order of occurrence in the regular expression.

{parts,N} - Specifies the number of parts the subject string is to be split into.

The number of parts is to be a positive integer for a specific maximum number of parts, and infinity for the maximum number of parts possible (the default). Specifying {parts,0} gives as many parts as possible disregarding empty parts at the end, the same as specifying trim.

trim - Specifies that empty parts at the end of the result list are to be disregarded. The same as specifying {parts,0}. This corresponds to the default behavior of the split built-in function in Perl.

version()

(since OTP 20.0)

-spec version() -> binary().

The return of this function is a string with the PCRE version of the system that was used in the Erlang/OTP compilation.

string

String processing functions.

This module provides functions for string processing.

A string in this module is represented by unicode:chardata/0, that is, a list of codepoints, binaries with UTF-8-encoded codepoints (UTF-8 binaries), or a mix of the two.

"abcd"               is a valid string
<<"abcd">>           is a valid string
["abcd"]             is a valid string
<<"abc..åäö"/utf8>>  is a valid string
<<"abc..åäö">>       is NOT a valid string,
                     but a binary with Latin-1-encoded codepoints
[<<"abc">>, "..åäö"] is a valid string
[atom]               is NOT a valid string

This module operates on grapheme clusters. A grapheme cluster is a user-perceived character, which can be represented by several codepoints.

"å"  [229] or [97, 778]
"e̊"  [101, 778]

The string length of "ß↑e̊" is 3, even though it is represented by the codepoints [223,8593,101,778] or the UTF-8 binary <<195,159,226,134,145,101,204,138>>.

Grapheme clusters for codepoints of class prepend and non-modern (or decomposed) Hangul is not handled for performance reasons in find/3, replace/3, split/2, split/3 and trim/3.

Splitting and appending strings is to be done on grapheme clusters borders. There is no verification that the results of appending strings are valid or normalized.

Most of the functions expect all input to be normalized to one form, see for example unicode:characters_to_nfc_list/1.

Language or locale specific handling of input is not considered in any function.

The functions can crash for non-valid input strings. For example, the functions expect UTF-8 binaries but not all functions verify that all binaries are encoded correctly.

Unless otherwise specified the return value type is the same as the input type. That is, binary input returns binary output, list input returns a list output, and mixed input can return a mixed output.

1> string:trim("  sarah  ").
"sarah"
2> string:trim(<<"  sarah  ">>).
<<"sarah">>
3> string:lexemes("foo bar", " ").
["foo","bar"]
4> string:lexemes(<<"foo bar">>, " ").
[<<"foo">>,<<"bar">>]

This module has been reworked in Erlang/OTP 20 to handle unicode:chardata/0 and operate on grapheme clusters. The old functions that only work on Latin-1 lists as input are still available but should not be used, they will be deprecated in a future release.

Notes

Some of the general string functions can seem to overlap each other. The reason is that this string package is the combination of two earlier packages and all functions of both packages have been retained.

Summary

Types

direction()

grapheme_cluster()

A user-perceived character, consisting of one or more codepoints.

Functions

casefold(String)

Converts String to a case-agnostic comparable string. Function casefold/1 is preferred over lowercase/1 when two strings are to be compared for equality. See also equal/4.

chomp(String)

Returns a string where any trailing \n or \r\n have been removed from String.

equal(A, B)

Equivalent to equal(A, B, true).

equal(A, B, IgnoreCase)

Equivalent to equal(A, B, IgnoreCase, none).

equal(A, B, IgnoreCase, Norm)

Returns true if A and B are equal, otherwise false.

find(String, SearchPattern)

Equivalent to find(String, SearchPattern, leading).

find(String, SearchPattern, Dir)

Removes anything before SearchPattern in String and returns the remainder of the string or nomatch if SearchPattern is not found. Dir, which can be leading or trailing, indicates from which direction characters are to be searched.

is_empty(String)

Returns true if String is the empty string, otherwise false.

jaro_similarity(String1, String2)

Returns a float between +0.0 and 1.0 representing the Jaro similarity between the given strings. Strings with a higher similarity will score closer to 1.0, with +0.0 meaning no similarity and 1.0 meaning an exact match.

length(String)

Returns the number of grapheme clusters in String.

lexemes(String, SeparatorList)

Returns a list of lexemes in String, separated by the grapheme clusters in SeparatorList.

lowercase(String)

Converts String to lowercase.

next_codepoint(String)

Returns the first codepoint in String and the rest of String in the tail. Returns an empty list if String is empty or an {error, String} tuple if the next byte is invalid.

next_grapheme(String)

Returns the first grapheme cluster in String and the rest of String in the tail. Returns an empty list if String is empty or an {error, String} tuple if the next byte is invalid.

nth_lexeme(String, N, SeparatorList)

Returns lexeme number N in String, where lexemes are separated by the grapheme clusters in SeparatorList.

pad(String, Length)

Equivalent to pad(String, Length, trailing).

pad(String, Length, Dir)

Equivalent to pad(String, Length, Dir, $ ).

pad(String, Length, Dir, Char)

Pads String to Length with grapheme cluster Char. Dir, which can be leading, trailing, or both, indicates where the padding should be added.

prefix(String, Prefix)

If Prefix is the prefix of String, removes it and returns the remainder of String, otherwise returns nomatch.

replace(String, SearchPattern, Replacement)

Equivalent to replace(String, SearchPattern, Replacement, leading).

replace(String, SearchPattern, Replacement, Where)

Replaces SearchPattern in String with Replacement. Where, indicates whether the leading, the trailing or all encounters of SearchPattern are to be replaced.

reverse(String)

Returns the reverse list of the grapheme clusters in String.

slice(String, Start)

Equivalent to slice(String, Start, infinity).

slice(String, Start, Length)

Returns a substring of String of at most Length grapheme clusters, starting at position Start.

split(String, SearchPattern)

Equivalent to split(String, SearchPattern, leading).

split(String, SearchPattern, Where)

Splits String where SearchPattern is encountered and return the remaining parts. Where, default leading, indicates whether the leading, the trailing or all encounters of SearchPattern will split String.

take(String, Characters)

Equivalent to take(String, Characters, false).

take(String, Characters, Complement)

Equivalent to take(String, Characters, Complement, leading).

take(String, Characters, Complement, Dir)

Takes characters from String as long as the characters are members of set Characters or the complement of set Characters. Dir, which can be leading or trailing, indicates from which direction characters are to be taken.

titlecase(String)

Converts String to titlecase.

to_float(String)

Argument String is expected to start with a valid text represented float (the digits are ASCII values). Remaining characters in the string after the float are returned in Rest.

to_graphemes(String)

Converts String to a list of grapheme clusters.

to_integer(String)

Argument String is expected to start with a valid text represented integer (the digits are ASCII values). Remaining characters in the string after the integer are returned in Rest.

trim(String)

Equivalent to trim(String, both).

trim(String, Dir)

Equivalent to trim(String, Dir, Whitespace}) where Whitespace is the set of nonbreakable whitespace codepoints, defined as Pattern_White_Space in Unicode Standard Annex #31.

trim(String, Dir, Characters)

Returns a string, where leading or trailing, or both, Characters have been removed.

uppercase(String)

Converts String to uppercase.

Obsolete API functions

centre(String, Number)

Equivalent to centre(String, Number, $ ).

centre(String, Number, Character)

Returns a string, where String is centered in the string and surrounded by blanks or Character. The resulting string has length Number.

chars(Character, Number)

Equivalent to chars(Character, Number, []).

chars(Character, Number, Tail)

Returns a string consisting of Number characters Character. Optionally, the string can end with string Tail.

chr(String, Character)

Returns the index of the first occurrence of Character in String. Returns 0 if Character does not occur.

concat(String1, String2)

Concatenates String1 and String2 to form a new string String3, which is returned.

copies(String, Number)

Returns a string containing String repeated Number times.

cspan(String, Chars)

Returns the length of the maximum initial segment of String, which consists entirely of characters not from Chars.

join(StringList, Separator)

Returns a string with the elements of StringList separated by the string in Separator.

left(String, Number)

Equivalent to left(String, Number, $ ).

left(String, Number, Character)

Returns String with the length adjusted in accordance with Number. The left margin is fixed. If length(String) < Number, then String is padded with blanks or Characters.

len(String)

Returns the number of characters in String.

rchr(String, Character)

Returns the index of the last occurrence of Character in String. Returns 0 if Character does not occur.

right(String, Number)

Equivalent to right(String, Number, $ ).

right(String, Number, Character)

Returns String with the length adjusted in accordance with Number. The right margin is fixed. If the length of (String) < Number, then String is padded with blanks or Characters.

rstr(String, SubString)

Returns the position where the last occurrence of SubString begins in String. Returns 0 if SubString does not exist in String.

span(String, Chars)

Returns the length of the maximum initial segment of String, which consists entirely of characters from Chars.

str(String, SubString)

Returns the position where the first occurrence of SubString begins in String. Returns 0 if SubString does not exist in String.

strip(String)

Equivalent to strip(String, both).

strip(String, Direction)

Equivalent to strip(String, Direction, $ ).

strip(String, Direction, Character)

Returns a string, where leading or trailing, or both, blanks or a number of Character have been removed.

sub_string(String, Start)

Equivalent to sub_string(String, Start, string:length(String)).

sub_string(String, Start, Stop)

Returns a substring of String, starting at position Start to the end of the string, or to and including position Stop.

sub_word(String, Number)

Equivalent to sub_word(String, Number, $ ).

sub_word(String, Number, Character)

Returns the word in position Number of String. Words are separated by blanks or Characters.

substr(String, Start)

Equivalent to substr(String, Start, string:length(String) - Start).

substr(String, Start, Length)

Returns a substring of String, starting at position Start, and ending at the end of the string or at length Length.

to_lower/1

The specified string or character is case-converted. Notice that the supported character set is ISO/IEC 8859-1 (also called Latin 1); all values outside this set are unchanged.

to_upper/1

The specified string or character is case-converted. Notice that the supported character set is ISO/IEC 8859-1 (also called Latin 1); all values outside this set are unchanged.

tokens(String, SeparatorList)

Returns a list of tokens in String, separated by the characters in SeparatorList.

words(String)

Equivalent to words(String, $ ).

words(String, Character)

Returns the number of words in String, separated by blanks or Character.

Types

direction()

(not exported)

-type direction() :: leading | trailing.

grapheme_cluster()

-type grapheme_cluster() :: char() | [char()].

A user-perceived character, consisting of one or more codepoints.

Functions

casefold(String)

(since OTP 20.0)

-spec casefold(String :: unicode:chardata()) -> unicode:chardata().

Converts String to a case-agnostic comparable string. Function casefold/1 is preferred over lowercase/1 when two strings are to be compared for equality. See also equal/4.

Example:

1> string:casefold("Ω and ẞ SHARP S").
"ω and ss sharp s"

chomp(String)

(since OTP 20.0)

-spec chomp(String :: unicode:chardata()) -> unicode:chardata().

Returns a string where any trailing \n or \r\n have been removed from String.

Example:

182> string:chomp(<<"\nHello\n\n">>).
<<"\nHello">>
183> string:chomp("\nHello\r\r\n").
"\nHello\r"

equal(A, B)

-spec equal(A, B) -> boolean() when A :: unicode:chardata(), B :: unicode:chardata().

Equivalent to equal(A, B, true).

equal(A, B, IgnoreCase)

(since OTP 20.0)

-spec equal(A, B, IgnoreCase) -> boolean()
               when A :: unicode:chardata(), B :: unicode:chardata(), IgnoreCase :: boolean().

Equivalent to equal(A, B, IgnoreCase, none).

equal(A, B, IgnoreCase, Norm)

(since OTP 20.0)

-spec equal(A, B, IgnoreCase, Norm) -> boolean()
               when
                   A :: unicode:chardata(),
                   B :: unicode:chardata(),
                   IgnoreCase :: boolean(),
                   Norm :: none | nfc | nfd | nfkc | nfkd.

Returns true if A and B are equal, otherwise false.

If IgnoreCase is true the function does casefolding on the fly before the equality test.

If Norm is not none the function applies normalization on the fly before the equality test. There are four available normalization forms: nfc, nfd, nfkc, and nfkd.

Example:

1> string:equal("åäö", <<"åäö"/utf8>>).
true
2> string:equal("åäö", unicode:characters_to_nfd_binary("åäö")).
false
3> string:equal("åäö", unicode:characters_to_nfd_binary("ÅÄÖ"), true, nfc).
true

find(String, SearchPattern)

(since OTP 20.0)

-spec find(String, SearchPattern) -> unicode:chardata() | nomatch
              when String :: unicode:chardata(), SearchPattern :: unicode:chardata().

Equivalent to find(String, SearchPattern, leading).

find(String, SearchPattern, Dir)

(since OTP 20.0)

-spec find(String, SearchPattern, Dir) -> unicode:chardata() | nomatch
              when String :: unicode:chardata(), SearchPattern :: unicode:chardata(), Dir :: direction().

Removes anything before SearchPattern in String and returns the remainder of the string or nomatch if SearchPattern is not found. Dir, which can be leading or trailing, indicates from which direction characters are to be searched.

Example:

1> string:find("ab..cd..ef", ".").
"..cd..ef"
2> string:find(<<"ab..cd..ef">>, "..", trailing).
<<"..ef">>
3> string:find(<<"ab..cd..ef">>, "x", leading).
nomatch
4> string:find("ab..cd..ef", "x", trailing).
nomatch

is_empty(String)

(since OTP 20.0)

-spec is_empty(String :: unicode:chardata()) -> boolean().

Returns true if String is the empty string, otherwise false.

Example:

1> string:is_empty("foo").
false
2> string:is_empty(["",<<>>]).
true

jaro_similarity(String1, String2)

(since OTP 27.0)

-spec jaro_similarity(String1, String2) -> Similarity
                         when
                             String1 :: unicode:chardata(),
                             String2 :: unicode:chardata(),
                             Similarity :: float().

Returns a float between +0.0 and 1.0 representing the Jaro similarity between the given strings. Strings with a higher similarity will score closer to 1.0, with +0.0 meaning no similarity and 1.0 meaning an exact match.

Example:

1> string:jaro_similarity("ditto", "ditto").
1.0
2> string:jaro_similarity("foo", "bar").
+0.0
3> string:jaro_similarity("michelle", "michael").
0.8690476190476191
4> string:jaro_similarity(<<"Édouard"/utf8>>, <<"Claude">>).
0.5317460317460317

The Jaro distance between two strings can be calculated with JaroDistance = 1.0 - JaroSimilarity.

length(String)

(since OTP 20.0)

-spec length(String :: unicode:chardata()) -> non_neg_integer().

Returns the number of grapheme clusters in String.

Example:

1> string:length("ß↑e̊").
3
2> string:length(<<195,159,226,134,145,101,204,138>>).
3

lexemes(String, SeparatorList)

(since OTP 20.0)

-spec lexemes(String :: unicode:chardata(), SeparatorList :: [grapheme_cluster()]) ->
                 [unicode:chardata()].

Returns a list of lexemes in String, separated by the grapheme clusters in SeparatorList.

Notice that, as shown in this example, two or more adjacent separator graphemes clusters in String are treated as one. That is, there are no empty strings in the resulting list of lexemes. See also split/3 which returns empty strings.

Notice that [$\r,$\n] is one grapheme cluster.

Example:

1> string:lexemes("abc de̊fxxghix jkl\r\nfoo", "x e" ++ [[$\r,$\n]]).
["abc","de̊f","ghi","jkl","foo"]
2> string:lexemes(<<"abc de̊fxxghix jkl\r\nfoo"/utf8>>, "x e" ++ [$\r,$\n]).
[<<"abc">>,<<"de̊f"/utf8>>,<<"ghi">>,<<"jkl\r\nfoo">>]

lowercase(String)

(since OTP 20.0)

-spec lowercase(String :: unicode:chardata()) -> unicode:chardata().

Converts String to lowercase.

Notice that function casefold/1 should be used when converting a string to be tested for equality.

Example:

2> string:lowercase(string:uppercase("Michał")).
"michał"

next_codepoint(String)

(since OTP 20.0)

-spec next_codepoint(String :: unicode:chardata()) ->
                        maybe_improper_list(char(), unicode:chardata()) | {error, unicode:chardata()}.

Returns the first codepoint in String and the rest of String in the tail. Returns an empty list if String is empty or an {error, String} tuple if the next byte is invalid.

Example:

1> string:next_codepoint(unicode:characters_to_binary("e̊fg")).
[101|<<"̊fg"/utf8>>]

next_grapheme(String)

(since OTP 20.0)

-spec next_grapheme(String :: unicode:chardata()) ->
                       maybe_improper_list(grapheme_cluster(), unicode:chardata()) |
                       {error, unicode:chardata()}.

Returns the first grapheme cluster in String and the rest of String in the tail. Returns an empty list if String is empty or an {error, String} tuple if the next byte is invalid.

Example:

1> string:next_grapheme(unicode:characters_to_binary("e̊fg")).
["e̊"|<<"fg">>]

nth_lexeme(String, N, SeparatorList)

(since OTP 20.0)

-spec nth_lexeme(String, N, SeparatorList) -> unicode:chardata()
                    when
                        String :: unicode:chardata(),
                        N :: non_neg_integer(),
                        SeparatorList :: [grapheme_cluster()].

Returns lexeme number N in String, where lexemes are separated by the grapheme clusters in SeparatorList.

Example:

1> string:nth_lexeme("abc.de̊f.ghiejkl", 3, ".e").
"ghi"

pad(String, Length)

(since OTP 20.0)

-spec pad(String, Length) -> unicode:charlist() when String :: unicode:chardata(), Length :: integer().

Equivalent to pad(String, Length, trailing).

pad(String, Length, Dir)

(since OTP 20.0)

-spec pad(String, Length, Dir) -> unicode:charlist()
             when String :: unicode:chardata(), Length :: integer(), Dir :: direction() | both.

Equivalent to pad(String, Length, Dir, $ ).

pad(String, Length, Dir, Char)

(since OTP 20.0)

-spec pad(String, Length, Dir, Char) -> unicode:charlist()
             when
                 String :: unicode:chardata(),
                 Length :: integer(),
                 Dir :: direction() | both,
                 Char :: grapheme_cluster().

Pads String to Length with grapheme cluster Char. Dir, which can be leading, trailing, or both, indicates where the padding should be added.

Example:

1> string:pad(<<"He̊llö"/utf8>>, 8).
[<<72,101,204,138,108,108,195,182>>,32,32,32]
2> io:format("'~ts'~n",[string:pad("He̊llö", 8, leading)]).
'   He̊llö'
3> io:format("'~ts'~n",[string:pad("He̊llö", 8, both)]).
' He̊llö  '

prefix(String, Prefix)

(since OTP 20.0)

-spec prefix(String :: unicode:chardata(), Prefix :: unicode:chardata()) -> nomatch | unicode:chardata().

If Prefix is the prefix of String, removes it and returns the remainder of String, otherwise returns nomatch.

Example:

1> string:prefix(<<"prefix of string">>, "pre").
<<"fix of string">>
2> string:prefix("pre", "prefix").
nomatch

replace(String, SearchPattern, Replacement)

(since OTP 20.0)

-spec replace(String, SearchPattern, Replacement) -> [unicode:chardata()]
                 when
                     String :: unicode:chardata(),
                     SearchPattern :: unicode:chardata(),
                     Replacement :: unicode:chardata().

Equivalent to replace(String, SearchPattern, Replacement, leading).

replace(String, SearchPattern, Replacement, Where)

(since OTP 20.0)

-spec replace(String, SearchPattern, Replacement, Where) -> [unicode:chardata()]
                 when
                     String :: unicode:chardata(),
                     SearchPattern :: unicode:chardata(),
                     Replacement :: unicode:chardata(),
                     Where :: direction() | all.

Replaces SearchPattern in String with Replacement. Where, indicates whether the leading, the trailing or all encounters of SearchPattern are to be replaced.

Can be implemented as:

lists:join(Replacement, split(String, SearchPattern, Where)).

Example:

1> string:replace(<<"ab..cd..ef">>, "..", "*").
[<<"ab">>,"*",<<"cd..ef">>]
2> string:replace(<<"ab..cd..ef">>, "..", "*", all).
[<<"ab">>,"*",<<"cd">>,"*",<<"ef">>]

reverse(String)

(since OTP 20.0)

-spec reverse(String :: unicode:chardata()) -> [grapheme_cluster()].

Returns the reverse list of the grapheme clusters in String.

Example:

1> Reverse = string:reverse(unicode:characters_to_nfd_binary("ÅÄÖ")).
[[79,776],[65,776],[65,778]]
2> io:format("~ts~n",[Reverse]).
ÖÄÅ

slice(String, Start)

(since OTP 20.0)

-spec slice(String, Start) -> Slice
               when
                   String :: unicode:chardata(), Start :: non_neg_integer(), Slice :: unicode:chardata().

Equivalent to slice(String, Start, infinity).

slice(String, Start, Length)

(since OTP 20.0)

-spec slice(String, Start, Length) -> Slice
               when
                   String :: unicode:chardata(),
                   Start :: non_neg_integer(),
                   Length :: infinity | non_neg_integer(),
                   Slice :: unicode:chardata().

Returns a substring of String of at most Length grapheme clusters, starting at position Start.

Example:

1> string:slice(<<"He̊llö Wörld"/utf8>>, 4).
<<"ö Wörld"/utf8>>
2> string:slice(["He̊llö ", <<"Wörld"/utf8>>], 4,4).
"ö Wö"
3> string:slice(["He̊llö ", <<"Wörld"/utf8>>], 4,50).
"ö Wörld"

split(String, SearchPattern)

(since OTP 20.0)

-spec split(String, SearchPattern) -> [unicode:chardata()]
               when String :: unicode:chardata(), SearchPattern :: unicode:chardata().

Equivalent to split(String, SearchPattern, leading).

split(String, SearchPattern, Where)

(since OTP 20.0)

-spec split(String, SearchPattern, Where) -> [unicode:chardata()]
               when
                   String :: unicode:chardata(),
                   SearchPattern :: unicode:chardata(),
                   Where :: direction() | all.

Splits String where SearchPattern is encountered and return the remaining parts. Where, default leading, indicates whether the leading, the trailing or all encounters of SearchPattern will split String.

Example:

0> string:split("ab..bc..cd", "..").
["ab","bc..cd"]
1> string:split(<<"ab..bc..cd">>, "..", trailing).
[<<"ab..bc">>,<<"cd">>]
2> string:split(<<"ab..bc....cd">>, "..", all).
[<<"ab">>,<<"bc">>,<<>>,<<"cd">>]

take(String, Characters)

(since OTP 20.0)

-spec take(String, Characters) -> {Leading, Trailing}
              when
                  String :: unicode:chardata(),
                  Characters :: [grapheme_cluster()],
                  Leading :: unicode:chardata(),
                  Trailing :: unicode:chardata().

Equivalent to take(String, Characters, false).

take(String, Characters, Complement)

(since OTP 20.0)

-spec take(String, Characters, Complement) -> {Leading, Trailing}
              when
                  String :: unicode:chardata(),
                  Characters :: [grapheme_cluster()],
                  Complement :: boolean(),
                  Leading :: unicode:chardata(),
                  Trailing :: unicode:chardata().

Equivalent to take(String, Characters, Complement, leading).

take(String, Characters, Complement, Dir)

(since OTP 20.0)

-spec take(String, Characters, Complement, Dir) -> {Leading, Trailing}
              when
                  String :: unicode:chardata(),
                  Characters :: [grapheme_cluster()],
                  Complement :: boolean(),
                  Dir :: direction(),
                  Leading :: unicode:chardata(),
                  Trailing :: unicode:chardata().

Takes characters from String as long as the characters are members of set Characters or the complement of set Characters. Dir, which can be leading or trailing, indicates from which direction characters are to be taken.

Example:

5> string:take("abc0z123", lists:seq($a,$z)).
{"abc","0z123"}
6> string:take(<<"abc0z123">>, lists:seq($0,$9), true, leading).
{<<"abc">>,<<"0z123">>}
7> string:take("abc0z123", lists:seq($0,$9), false, trailing).
{"abc0z","123"}
8> string:take(<<"abc0z123">>, lists:seq($a,$z), true, trailing).
{<<"abc0z">>,<<"123">>}

titlecase(String)

(since OTP 20.0)

-spec titlecase(String :: unicode:chardata()) -> unicode:chardata().

Converts String to titlecase.

Example:

1> string:titlecase("ß is a SHARP s").
"Ss is a SHARP s"

to_float(String)

-spec to_float(String) -> {Float, Rest} | {error, Reason}
                  when
                      String :: unicode:chardata(),
                      Float :: float(),
                      Rest :: unicode:chardata(),
                      Reason :: no_float | badarg.

Argument String is expected to start with a valid text represented float (the digits are ASCII values). Remaining characters in the string after the float are returned in Rest.

Example:

1> {F1,Fs} = string:to_float("1.0-1.0e-1"),
1> {F2,[]} = string:to_float(Fs),
1> F1+F2.
0.9
2> string:to_float("3/2=1.5").
{error,no_float}
3> string:to_float("-1.5eX").
{-1.5,"eX"}

to_graphemes(String)

(since OTP 20.0)

-spec to_graphemes(String :: unicode:chardata()) -> [grapheme_cluster()].

Converts String to a list of grapheme clusters.

Example:

1> string:to_graphemes("ß↑e̊").
[223,8593,[101,778]]
2> string:to_graphemes(<<"ß↑e̊"/utf8>>).
[223,8593,[101,778]]

to_integer(String)

-spec to_integer(String) -> {Int, Rest} | {error, Reason}
                    when
                        String :: unicode:chardata(),
                        Int :: integer(),
                        Rest :: unicode:chardata(),
                        Reason :: no_integer | badarg.

Argument String is expected to start with a valid text represented integer (the digits are ASCII values). Remaining characters in the string after the integer are returned in Rest.

Example:

1> {I1,Is} = string:to_integer("33+22"),
1> {I2,[]} = string:to_integer(Is),
1> I1-I2.
11
2> string:to_integer("0.5").
{0,".5"}
3> string:to_integer("x=2").
{error,no_integer}

trim(String)

(since OTP 20.0)

-spec trim(String) -> unicode:chardata() when String :: unicode:chardata().

Equivalent to trim(String, both).

trim(String, Dir)

(since OTP 20.0)

-spec trim(String, Dir) -> unicode:chardata()
              when String :: unicode:chardata(), Dir :: direction() | both.

Equivalent to trim(String, Dir, Whitespace}) where Whitespace is the set of nonbreakable whitespace codepoints, defined as Pattern_White_Space in Unicode Standard Annex #31.

trim(String, Dir, Characters)

(since OTP 20.0)

-spec trim(String, Dir, Characters) -> unicode:chardata()
              when
                  String :: unicode:chardata(),
                  Dir :: direction() | both,
                  Characters :: [grapheme_cluster()].

Returns a string, where leading or trailing, or both, Characters have been removed.

Dir which can be leading, trailing, or both, indicates from which direction characters are to be removed.

Note that [$\r,$\n] is one grapheme cluster according to the Unicode Standard.

Example:

1> string:trim("\t  Hello  \n").
"Hello"
2> string:trim(<<"\t  Hello  \n">>, leading).
<<"Hello  \n">>
3> string:trim(<<".Hello.\n">>, trailing, "\n.").
<<".Hello">>

uppercase(String)

(since OTP 20.0)

-spec uppercase(String :: unicode:chardata()) -> unicode:chardata().

Converts String to uppercase.

See also titlecase/1.

Example:

1> string:uppercase("Michał").
"MICHAŁ"

Obsolete API functions

centre(String, Number)

-spec centre(String, Number) -> Centered
                when String :: string(), Centered :: string(), Number :: non_neg_integer().

Equivalent to centre(String, Number, $ ).

centre(String, Number, Character)

-spec centre(String, Number, Character) -> Centered
                when
                    String :: string(),
                    Centered :: string(),
                    Number :: non_neg_integer(),
                    Character :: char().

Returns a string, where String is centered in the string and surrounded by blanks or Character. The resulting string has length Number.

This function is obsolete. Use pad/3.

chars(Character, Number)

-spec chars(Character, Number) -> String
               when Character :: char(), Number :: non_neg_integer(), String :: string().

Equivalent to chars(Character, Number, []).

chars(Character, Number, Tail)

-spec chars(Character, Number, Tail) -> String
               when
                   Character :: char(),
                   Number :: non_neg_integer(),
                   Tail :: string(),
                   String :: string().

Returns a string consisting of Number characters Character. Optionally, the string can end with string Tail.

This function is obsolete. Use lists:duplicate/2.

chr(String, Character)

-spec chr(String, Character) -> Index
             when String :: string(), Character :: char(), Index :: non_neg_integer().

Returns the index of the first occurrence of Character in String. Returns 0 if Character does not occur.

This function is obsolete. Use find/2.

concat(String1, String2)

-spec concat(String1, String2) -> String3
                when String1 :: string(), String2 :: string(), String3 :: string().

Concatenates String1 and String2 to form a new string String3, which is returned.

This function is obsolete. Use [String1, String2] as Data argument, and call unicode:characters_to_list/2 or unicode:characters_to_binary/2 to flatten the output.

copies(String, Number)

-spec copies(String, Number) -> Copies
                when String :: string(), Copies :: string(), Number :: non_neg_integer().

Returns a string containing String repeated Number times.

This function is obsolete. Use lists:duplicate/2.

cspan(String, Chars)

-spec cspan(String, Chars) -> Length
               when String :: string(), Chars :: string(), Length :: non_neg_integer().

Returns the length of the maximum initial segment of String, which consists entirely of characters not from Chars.

This function is obsolete. Use take/3.

Example:

1> string:cspan("\t    abcdef", " \t").
0

join(StringList, Separator)

-spec join(StringList, Separator) -> String
              when StringList :: [string()], Separator :: string(), String :: string().

Returns a string with the elements of StringList separated by the string in Separator.

This function is obsolete. Use lists:join/2.

Example:

1> join(["one", "two", "three"], ", ").
"one, two, three"

left(String, Number)

-spec left(String, Number) -> Left
              when String :: string(), Left :: string(), Number :: non_neg_integer().

Equivalent to left(String, Number, $ ).

left(String, Number, Character)

-spec left(String, Number, Character) -> Left
              when
                  String :: string(), Left :: string(), Number :: non_neg_integer(), Character :: char().

Returns String with the length adjusted in accordance with Number. The left margin is fixed. If length(String) < Number, then String is padded with blanks or Characters.

This function is obsolete. Use pad/2 or pad/3.

Example:

1> string:left("Hello",10,$.).
"Hello....."

len(String)

-spec len(String) -> Length when String :: string(), Length :: non_neg_integer().

Returns the number of characters in String.

This function is obsolete. Use length/1.

rchr(String, Character)

-spec rchr(String, Character) -> Index
              when String :: string(), Character :: char(), Index :: non_neg_integer().

Returns the index of the last occurrence of Character in String. Returns 0 if Character does not occur.

This function is obsolete. Use find/3.

right(String, Number)

-spec right(String, Number) -> Right
               when String :: string(), Right :: string(), Number :: non_neg_integer().

Equivalent to right(String, Number, $ ).

right(String, Number, Character)

-spec right(String, Number, Character) -> Right
               when
                   String :: string(),
                   Right :: string(),
                   Number :: non_neg_integer(),
                   Character :: char().

Returns String with the length adjusted in accordance with Number. The right margin is fixed. If the length of (String) < Number, then String is padded with blanks or Characters.

This function is obsolete. Use pad/3.

Example:

1> string:right("Hello", 10, $.).
".....Hello"

rstr(String, SubString)

-spec rstr(String, SubString) -> Index
              when String :: string(), SubString :: string(), Index :: non_neg_integer().

Returns the position where the last occurrence of SubString begins in String. Returns 0 if SubString does not exist in String.

This function is obsolete. Use find/3.

Example:

1> string:rstr(" Hello Hello World World ", "Hello World").
8

span(String, Chars)

-spec span(String, Chars) -> Length
              when String :: string(), Chars :: string(), Length :: non_neg_integer().

Returns the length of the maximum initial segment of String, which consists entirely of characters from Chars.

This function is obsolete. Use take/2.

Example:

1> string:span("\t    abcdef", " \t").
5

str(String, SubString)

-spec str(String, SubString) -> Index
             when String :: string(), SubString :: string(), Index :: non_neg_integer().

Returns the position where the first occurrence of SubString begins in String. Returns 0 if SubString does not exist in String.

This function is obsolete. Use find/2.

Example:

1> string:str(" Hello Hello World World ", "Hello World").
8

strip(String)

-spec strip(string()) -> string().

Equivalent to strip(String, both).

strip(String, Direction)

-spec strip(String, Direction) -> Stripped
               when String :: string(), Stripped :: string(), Direction :: left | right | both.

Equivalent to strip(String, Direction, $ ).

strip(String, Direction, Character)

-spec strip(String, Direction, Character) -> Stripped
               when
                   String :: string(),
                   Stripped :: string(),
                   Direction :: left | right | both,
                   Character :: char().

Returns a string, where leading or trailing, or both, blanks or a number of Character have been removed.

Direction, which can be left, right, or both, indicates from which direction blanks are to be removed. strip/1 is equivalent to strip(String, both).

This function is obsolete. Use trim/3.

Example:

1> string:strip("...Hello.....", both, $.).
"Hello"

sub_string(String, Start)

-spec sub_string(String, Start) -> SubString
                    when String :: string(), SubString :: string(), Start :: pos_integer().

Equivalent to sub_string(String, Start, string:length(String)).

sub_string(String, Start, Stop)

-spec sub_string(String, Start, Stop) -> SubString
                    when
                        String :: string(),
                        SubString :: string(),
                        Start :: pos_integer(),
                        Stop :: pos_integer().

Returns a substring of String, starting at position Start to the end of the string, or to and including position Stop.

This function is obsolete. Use slice/3.

Example:

1> sub_string("Hello World", 4, 8).
"lo Wo"

sub_word(String, Number)

-spec sub_word(String, Number) -> Word when String :: string(), Word :: string(), Number :: integer().

Equivalent to sub_word(String, Number, $ ).

sub_word(String, Number, Character)

-spec sub_word(String, Number, Character) -> Word
                  when String :: string(), Word :: string(), Number :: integer(), Character :: char().

Returns the word in position Number of String. Words are separated by blanks or Characters.

This function is obsolete. Use nth_lexeme/3.

Example:

1> string:sub_word(" Hello old boy !",3,$o).
"ld b"

substr(String, Start)

-spec substr(String, Start) -> SubString
                when String :: string(), SubString :: string(), Start :: pos_integer().

Equivalent to substr(String, Start, string:length(String) - Start).

substr(String, Start, Length)

-spec substr(String, Start, Length) -> SubString
                when
                    String :: string(),
                    SubString :: string(),
                    Start :: pos_integer(),
                    Length :: non_neg_integer().

Returns a substring of String, starting at position Start, and ending at the end of the string or at length Length.

This function is obsolete. Use slice/3.

Example:

1> substr("Hello World", 4, 5).
"lo Wo"

to_lower/1

-spec to_lower(String) -> Result when String :: io_lib:latin1_string(), Result :: io_lib:latin1_string();
              (Char) -> CharResult when Char :: char(), CharResult :: char().

The specified string or character is case-converted. Notice that the supported character set is ISO/IEC 8859-1 (also called Latin 1); all values outside this set are unchanged.

This function is obsolete use lowercase/1, titlecase/1 or casefold/1.

to_upper/1

-spec to_upper(String) -> Result when String :: io_lib:latin1_string(), Result :: io_lib:latin1_string();
              (Char) -> CharResult when Char :: char(), CharResult :: char().

The specified string or character is case-converted. Notice that the supported character set is ISO/IEC 8859-1 (also called Latin 1); all values outside this set are unchanged.

This function is obsolete use uppercase/1, titlecase/1 or casefold/1.

tokens(String, SeparatorList)

-spec tokens(String, SeparatorList) -> Tokens
                when
                    String :: string(),
                    SeparatorList :: string(),
                    Tokens :: [Token :: nonempty_string()].

Returns a list of tokens in String, separated by the characters in SeparatorList.

Example:

1> tokens("abc defxxghix jkl", "x ").
["abc", "def", "ghi", "jkl"]

Notice that, as shown in this example, two or more adjacent separator characters in String are treated as one. That is, there are no empty strings in the resulting list of tokens.

This function is obsolete. Use lexemes/2.

words(String)

-spec words(String) -> Count when String :: string(), Count :: pos_integer().

Equivalent to words(String, $ ).

words(String, Character)

-spec words(String, Character) -> Count
               when String :: string(), Character :: char(), Count :: pos_integer().

Returns the number of words in String, separated by blanks or Character.

This function is obsolete. Use lexemes/2.

Example:

1> words(" Hello old boy!", $o).
4

unicode

Functions for converting Unicode characters.

This module contains functions for converting between different character representations. It converts between ISO Latin-1 characters and Unicode characters, but it can also convert between different Unicode encodings (like UTF-8, UTF-16, and UTF-32).

The default Unicode encoding in Erlang binaries is UTF-8, which is also the format in which built-in functions and libraries in OTP expect to find binary Unicode data. In lists, Unicode data is encoded as integers, each integer representing one character and encoded simply as the Unicode code point for the character.

Other Unicode encodings than integers representing code points or UTF-8 in binaries are referred to as "external encodings". The ISO Latin-1 encoding is in binaries and lists referred to as latin1-encoding.

It is recommended to only use external encodings for communication with external entities where this is required. When working inside the Erlang/OTP environment, it is recommended to keep binaries in UTF-8 when representing Unicode characters. ISO Latin-1 encoding is supported both for backward compatibility and for communication with external entities not supporting Unicode character sets.

Programs should always operate on a normalized form and compare canonical-equivalent Unicode characters as equal. All characters should thus be normalized to one form once on the system borders. One of the following functions can convert characters to their normalized forms characters_to_nfc_list/1, characters_to_nfc_binary/1, characters_to_nfd_list/1 or characters_to_nfd_binary/1. For general text characters_to_nfc_list/1 or characters_to_nfc_binary/1 is preferred, and for identifiers one of the compatibility normalization functions, such as characters_to_nfkc_list/1, is preferred for security reasons. The normalization functions where introduced in OTP 20. Additional information on normalization can be found in the Unicode FAQ.

Summary

Types

chardata()

charlist()

encoding()

endian()

external_chardata()

external_charlist()

external_unicode_binary()

A binary/0 with characters coded in a user-specified Unicode encoding other than UTF-8 (that is, UTF-16 or UTF-32).

latin1_binary()

A binary/0 with characters coded in ISO Latin-1.

latin1_char()

An integer/0 representing a valid ISO Latin-1 character (0-255).

latin1_chardata()

Equivalent to iodata/0.

latin1_charlist()

Equivalent to iolist/0.

unicode_binary()

A binary/0 with characters encoded in the UTF-8 coding standard.

Functions

bom_to_encoding(Bin)

Checks for a UTF Byte Order Mark (BOM) in the beginning of a binary.

characters_to_binary(Data)

Equivalent to characters_to_binary(Data, unicode, unicode).

characters_to_binary(Data, InEncoding)

Equivalent to characters_to_binary(Data, InEncoding, unicode).

characters_to_binary(Data, InEncoding, OutEncoding)

Behaves as characters_to_list/2, but produces a binary instead of a Unicode list.

characters_to_list(Data)

Equivalent to characters_to_list(Data, unicode).

characters_to_list(Data, InEncoding)

Converts a possibly deep list of integers and binaries into a list of integers representing Unicode characters. The binaries in the input can have characters encoded as one of the following

characters_to_nfc_binary(CD)

Converts a possibly deep list of characters and binaries into a Normalized Form of canonical equivalent Composed characters according to the Unicode standard.

characters_to_nfc_list(CD)

Converts a possibly deep list of characters and binaries into a Normalized Form of canonical equivalent Composed characters according to the Unicode standard.

characters_to_nfd_binary(CD)

Converts a possibly deep list of characters and binaries into a Normalized Form of canonical equivalent Decomposed characters according to the Unicode standard.

characters_to_nfd_list(CD)

Converts a possibly deep list of characters and binaries into a Normalized Form of canonical equivalent Decomposed characters according to the Unicode standard.

characters_to_nfkc_binary(CD)

Converts a possibly deep list of characters and binaries into a Normalized Form of compatibly equivalent Composed characters according to the Unicode standard.

characters_to_nfkc_list(CD)

Converts a possibly deep list of characters and binaries into a Normalized Form of compatibly equivalent Composed characters according to the Unicode standard.

characters_to_nfkd_binary(CD)

Converts a possibly deep list of characters and binaries into a Normalized Form of compatibly equivalent Decomposed characters according to the Unicode standard.

characters_to_nfkd_list(CD)

Converts a possibly deep list of characters and binaries into a Normalized Form of compatibly equivalent Decomposed characters according to the Unicode standard.

encoding_to_bom(InEncoding)

Creates a UTF Byte Order Mark (BOM) as a binary from the supplied InEncoding.

Types

chardata()

-type chardata() :: charlist() | unicode_binary().

charlist()

-type charlist() :: maybe_improper_list(char() | unicode_binary() | charlist(), unicode_binary() | []).

encoding()

-type encoding() :: latin1 | unicode | utf8 | utf16 | {utf16, endian()} | utf32 | {utf32, endian()}.

endian()

(not exported)

-type endian() :: big | little.

external_chardata()

-type external_chardata() :: external_charlist() | external_unicode_binary().

external_charlist()

-type external_charlist() ::
          maybe_improper_list(char() | external_unicode_binary() | external_charlist(),
                              external_unicode_binary() | []).

external_unicode_binary()

(not exported)

-type external_unicode_binary() :: binary().

A binary/0 with characters coded in a user-specified Unicode encoding other than UTF-8 (that is, UTF-16 or UTF-32).

latin1_binary()

-type latin1_binary() :: binary().

A binary/0 with characters coded in ISO Latin-1.

latin1_char()

-type latin1_char() :: byte().

An integer/0 representing a valid ISO Latin-1 character (0-255).

latin1_chardata()

-type latin1_chardata() :: latin1_charlist() | latin1_binary().

Equivalent to iodata/0.

latin1_charlist()

-type latin1_charlist() ::
          maybe_improper_list(latin1_char() | latin1_binary() | latin1_charlist(), latin1_binary() | []).

Equivalent to iolist/0.

unicode_binary()

-type unicode_binary() :: binary().

A binary/0 with characters encoded in the UTF-8 coding standard.

Functions

bom_to_encoding(Bin)

-spec bom_to_encoding(Bin) -> {Encoding, Length}
                         when
                             Bin :: binary(),
                             Encoding :: latin1 | utf8 | {utf16, endian()} | {utf32, endian()},
                             Length :: non_neg_integer().

Checks for a UTF Byte Order Mark (BOM) in the beginning of a binary.

If the supplied binary Bin begins with a valid BOM for either UTF-8, UTF-16, or UTF-32, the function returns the encoding identified along with the BOM length in bytes.

If no BOM is found, the function returns {latin1,0}.

characters_to_binary(Data)

-spec characters_to_binary(Data) -> Result
                              when
                                  Data :: latin1_chardata() | chardata() | external_chardata(),
                                  Result ::
                                      binary() |
                                      {error, binary(), RestData} |
                                      {incomplete, binary(), binary()},
                                  RestData :: latin1_chardata() | chardata() | external_chardata().

Equivalent to characters_to_binary(Data, unicode, unicode).

characters_to_binary(Data, InEncoding)

-spec characters_to_binary(Data, InEncoding) -> Result
                              when
                                  Data :: latin1_chardata() | chardata() | external_chardata(),
                                  InEncoding :: encoding(),
                                  Result ::
                                      binary() |
                                      {error, binary(), RestData} |
                                      {incomplete, binary(), binary()},
                                  RestData :: latin1_chardata() | chardata() | external_chardata().

Equivalent to characters_to_binary(Data, InEncoding, unicode).

characters_to_binary(Data, InEncoding, OutEncoding)

-spec characters_to_binary(Data, InEncoding, OutEncoding) -> Result
                              when
                                  Data :: latin1_chardata() | chardata() | external_chardata(),
                                  InEncoding :: encoding(),
                                  OutEncoding :: encoding(),
                                  Result ::
                                      binary() |
                                      {error, binary(), RestData} |
                                      {incomplete, binary(), binary()},
                                  RestData :: latin1_chardata() | chardata() | external_chardata().

Behaves as characters_to_list/2, but produces a binary instead of a Unicode list.

InEncoding defines how input is to be interpreted if binaries are present in Data

OutEncoding defines in what format output is to be generated.

Options:

unicode - An alias for utf8, as this is the preferred encoding for Unicode characters in binaries.

utf16 - An alias for {utf16,big}.

utf32 - An alias for {utf32,big}.

The atoms big and little denote big- or little-endian encoding.

Errors and exceptions occur as in characters_to_list/2, but the second element in tuple error or incomplete is a binary/0 and not a list/0.

characters_to_list(Data)

-spec characters_to_list(Data) -> Result
                            when
                                Data :: latin1_chardata() | chardata() | external_chardata(),
                                Result ::
                                    string() |
                                    {error, string(), RestData} |
                                    {incomplete, string(), binary()},
                                RestData :: latin1_chardata() | chardata() | external_chardata().

Equivalent to characters_to_list(Data, unicode).

characters_to_list(Data, InEncoding)

-spec characters_to_list(Data, InEncoding) -> Result
                            when
                                Data :: latin1_chardata() | chardata() | external_chardata(),
                                InEncoding :: encoding(),
                                Result ::
                                    string() |
                                    {error, string(), RestData} |
                                    {incomplete, string(), binary()},
                                RestData :: latin1_chardata() | chardata() | external_chardata().

Converts a possibly deep list of integers and binaries into a list of integers representing Unicode characters. The binaries in the input can have characters encoded as one of the following:

ISO Latin-1 (0-255, one character per byte). Here, case parameter InEncoding is to be specified as latin1.

One of the UTF-encodings, which is specified as parameter InEncoding.

Note that integers in the list always represent code points regardless of InEncoding passed. If InEncoding latin1 is passed, only code points < 256 are allowed; otherwise, all valid unicode code points are allowed.

If InEncoding is latin1, parameter Data corresponds to the iodata/0 type, but for unicode, parameter Data can contain integers > 255 (Unicode characters beyond the ISO Latin-1 range), which makes it invalid as iodata/0.

The purpose of the function is mainly to convert combinations of Unicode characters into a pure Unicode string in list representation for further processing. For writing the data to an external entity, the reverse function characters_to_binary/3 comes in handy.

Option unicode is an alias for utf8, as this is the preferred encoding for Unicode characters in binaries. utf16 is an alias for {utf16,big} and utf32 is an alias for {utf32,big}. The atoms big and little denote big- or little-endian encoding.

If the data cannot be converted, either because of illegal Unicode/ISO Latin-1 characters in the list, or because of invalid UTF encoding in any binaries, an error tuple is returned. The error tuple contains the tag error, a list representing the characters that could be converted before the error occurred and a representation of the characters including and after the offending integer/bytes. The last part is mostly for debugging, as it still constitutes a possibly deep or mixed list, or both, not necessarily of the same depth as the original data. The error occurs when traversing the list and whatever is left to decode is returned "as is".

However, if the input Data is a pure binary, the third part of the error tuple is guaranteed to be a binary as well.

Errors occur for the following reasons:

Integers out of range.

If InEncoding is latin1, an error occurs whenever an integer > 255 is found in the lists.

If InEncoding is of a Unicode type, an error occurs whenever either of the following is found:

An integer > 16#10FFFF (the maximum Unicode character)

An integer in the range 16#D800 to 16#DFFF (invalid range reserved for UTF-16 surrogate pairs)

Incorrect UTF encoding.

If InEncoding is one of the UTF types, the bytes in any binaries must be valid in that encoding.

Errors can occur for various reasons, including the following:

"Pure" decoding errors (like the upper bits of the bytes being wrong).

The bytes are decoded to a too large number.

The bytes are decoded to a code point in the invalid Unicode range.

Encoding is "overlong", meaning that a number should have been encoded in fewer bytes.

The case of a truncated UTF is handled specially, see the paragraph about incomplete binaries below.

If InEncoding is latin1, binaries are always valid as long as they contain whole bytes, as each byte falls into the valid ISO Latin-1 range.

A special type of error is when no actual invalid integers or bytes are found, but a trailing binary/0 consists of too few bytes to decode the last character. This error can occur if bytes are read from a file in chunks or if binaries in other ways are split on non-UTF character boundaries. An incomplete tuple is then returned instead of the error tuple. It consists of the same parts as the error tuple, but the tag is incomplete instead of error and the last element is always guaranteed to be a binary consisting of the first part of a (so far) valid UTF character.

If one UTF character is split over two consecutive binaries in the Data, the conversion succeeds. This means that a character can be decoded from a range of binaries as long as the whole range is specified as input without errors occurring.

Example:

decode_data(Data) ->
   case unicode:characters_to_list(Data,unicode) of
      {incomplete,Encoded, Rest} ->
            More = get_some_more_data(),
            Encoded ++ decode_data([Rest, More]);
      {error,Encoded,Rest} ->
            handle_error(Encoded,Rest);
      List ->
            List
   end.

However, bit strings that are not whole bytes are not allowed, so a UTF character must be split along 8-bit boundaries to ever be decoded.

A badarg exception is thrown for the following cases:

Any parameters are of the wrong type.

The list structure is invalid (a number as tail).

The binaries do not contain whole bytes (bit strings).

characters_to_nfc_binary(CD)

(since OTP 20.0)

-spec characters_to_nfc_binary(chardata()) -> unicode_binary() | {error, unicode_binary(), chardata()}.

Converts a possibly deep list of characters and binaries into a Normalized Form of canonical equivalent Composed characters according to the Unicode standard.

Any binaries in the input must be encoded with utf8 encoding.

The result is an utf8 encoded binary.

4> unicode:characters_to_nfc_binary([<<"abc..a">>,[778],$a,[776],$o,[776]]).
<<"abc..åäö"/utf8>>

characters_to_nfc_list(CD)

(since OTP 20.0)

-spec characters_to_nfc_list(chardata()) -> [char()] | {error, [char()], chardata()}.

Converts a possibly deep list of characters and binaries into a Normalized Form of canonical equivalent Composed characters according to the Unicode standard.

Any binaries in the input must be encoded with utf8 encoding.

The result is a list of characters.

3> unicode:characters_to_nfc_list([<<"abc..a">>,[778],$a,[776],$o,[776]]).
"abc..åäö"

characters_to_nfd_binary(CD)

(since OTP 20.0)

-spec characters_to_nfd_binary(chardata()) -> unicode_binary() | {error, unicode_binary(), chardata()}.

Converts a possibly deep list of characters and binaries into a Normalized Form of canonical equivalent Decomposed characters according to the Unicode standard.

Any binaries in the input must be encoded with utf8 encoding.

The result is an utf8 encoded binary.

2> unicode:characters_to_nfd_binary("abc..åäö").
<<97,98,99,46,46,97,204,138,97,204,136,111,204,136>>

characters_to_nfd_list(CD)

(since OTP 20.0)

-spec characters_to_nfd_list(chardata()) -> [char()] | {error, [char()], chardata()}.

Converts a possibly deep list of characters and binaries into a Normalized Form of canonical equivalent Decomposed characters according to the Unicode standard.

Any binaries in the input must be encoded with utf8 encoding.

The result is a list of characters.

1> unicode:characters_to_nfd_list("abc..åäö").
[97,98,99,46,46,97,778,97,776,111,776]

characters_to_nfkc_binary(CD)

(since OTP 20.0)

-spec characters_to_nfkc_binary(chardata()) -> unicode_binary() | {error, unicode_binary(), chardata()}.

Converts a possibly deep list of characters and binaries into a Normalized Form of compatibly equivalent Composed characters according to the Unicode standard.

Any binaries in the input must be encoded with utf8 encoding.

The result is an utf8 encoded binary.

4> unicode:characters_to_nfkc_binary([<<"abc..a">>,[778],$a,[776],$o,[776],[65299,65298]]).
<<"abc..åäö32"/utf8>>

characters_to_nfkc_list(CD)

(since OTP 20.0)

-spec characters_to_nfkc_list(chardata()) -> [char()] | {error, [char()], chardata()}.

Converts a possibly deep list of characters and binaries into a Normalized Form of compatibly equivalent Composed characters according to the Unicode standard.

Any binaries in the input must be encoded with utf8 encoding.

The result is a list of characters.

3> unicode:characters_to_nfkc_list([<<"abc..a">>,[778],$a,[776],$o,[776],[65299,65298]]).
"abc..åäö32"

characters_to_nfkd_binary(CD)

(since OTP 20.0)

-spec characters_to_nfkd_binary(chardata()) -> unicode_binary() | {error, unicode_binary(), chardata()}.

Converts a possibly deep list of characters and binaries into a Normalized Form of compatibly equivalent Decomposed characters according to the Unicode standard.

Any binaries in the input must be encoded with utf8 encoding.

The result is an utf8 encoded binary.

2> unicode:characters_to_nfkd_binary(["abc..åäö",[65299,65298]]).
<<97,98,99,46,46,97,204,138,97,204,136,111,204,136,51,50>>

characters_to_nfkd_list(CD)

(since OTP 20.0)

-spec characters_to_nfkd_list(chardata()) -> [char()] | {error, [char()], chardata()}.

Converts a possibly deep list of characters and binaries into a Normalized Form of compatibly equivalent Decomposed characters according to the Unicode standard.

Any binaries in the input must be encoded with utf8 encoding.

The result is a list of characters.

1> unicode:characters_to_nfkd_list(["abc..åäö",[65299,65298]]).
[97,98,99,46,46,97,778,97,776,111,776,51,50]

encoding_to_bom(InEncoding)

-spec encoding_to_bom(InEncoding) -> Bin when Bin :: binary(), InEncoding :: encoding().

Creates a UTF Byte Order Mark (BOM) as a binary from the supplied InEncoding.

The BOM is, if supported at all, expected to be placed first in UTF encoded files or messages.

The function returns <<>> for latin1 encoding, as there is no BOM for ISO Latin-1.

Notice that the BOM for UTF-8 is seldom used, and it is really not a byte order mark. There are obviously no byte order issues with UTF-8, so the BOM is only there to differentiate UTF-8 encoding from other UTF formats.

uri_string

URI processing functions.

This module contains functions for parsing and handling URIs (RFC 3986) and form-urlencoded query strings (HTML 5.2).

Parsing and serializing non-UTF-8 form-urlencoded query strings are also supported (HTML 5.0).

A URI is an identifier consisting of a sequence of characters matching the syntax rule named URI in RFC 3986.

The generic URI syntax consists of a hierarchical sequence of components referred to as the scheme, authority, path, query, and fragment:

    URI         = scheme ":" hier-part [ "?" query ] [ "#" fragment ]
    hier-part   = "//" authority path-abempty
                   / path-absolute
                   / path-rootless
                   / path-empty
    scheme      = ALPHA *( ALPHA / DIGIT / "+" / "-" / "." )
    authority   = [ userinfo "@" ] host [ ":" port ]
    userinfo    = *( unreserved / pct-encoded / sub-delims / ":" )

    reserved    = gen-delims / sub-delims
    gen-delims  = ":" / "/" / "?" / "#" / "[" / "]" / "@"
    sub-delims  = "!" / "$" / "&" / "'" / "(" / ")"
                / "*" / "+" / "," / ";" / "="

    unreserved  = ALPHA / DIGIT / "-" / "." / "_" / "~"

The interpretation of a URI depends only on the characters used and not on how those characters are represented in a network protocol.

The functions implemented by this module cover the following use cases:

Parsing URIs into its components and returing a map: parse/1

Recomposing a map of URI components into a URI string: recompose/1

Changing inbound binary and percent-encoding of URIs: transcode/2

Transforming URIs into a normalized form: normalize/1, normalize/2

Composing form-urlencoded query strings from a list of key-value pairs: compose_query/1, compose_query/2

Dissecting form-urlencoded query strings into a list of key-value pairs: dissect_query/1

Decoding percent-encoded triplets in URI map or a specific component of URI: percent_decode/1

Preparing and retrieving application specific data included in URI components: quote/1, quote/2, unquote/1

There are four different encodings present during the handling of URIs:

Inbound binary encoding in binaries

Inbound percent-encoding in lists and binaries

Outbound binary encoding in binaries

Outbound percent-encoding in lists and binaries

Functions with uri_string/0 argument accept lists, binaries and mixed lists (lists with binary elements) as input type. All of the functions but transcode/2 expects input as lists of unicode codepoints, UTF-8 encoded binaries and UTF-8 percent-encoded URI parts ("%C3%B6" corresponds to the unicode character "ö").

Unless otherwise specified the return value type and encoding are the same as the input type and encoding. That is, binary input returns binary output, list input returns a list output but mixed input returns list output.

In case of lists there is only percent-encoding. In binaries, however, both binary encoding and percent-encoding shall be considered. transcode/2 provides the means to convert between the supported encodings, it takes a uri_string/0 and a list of options specifying inbound and outbound encodings.

RFC 3986 does not mandate any specific character encoding and it is usually defined by the protocol or surrounding text. This library takes the same assumption, binary and percent-encoding are handled as one configuration unit, they cannot be set to different values.

Quoting functions are intended to be used by URI producing application during component preparation or retrieval phase to avoid conflicts between data and characters used in URI syntax. Quoting functions use percent encoding, but with different rules than for example during execution of recompose/1. It is user responsibility to provide quoting functions with application data only and using their output to combine an URI component.
Quoting functions can for instance be used for constructing a path component with a segment containing '/' character which should not collide with '/' used as general delimiter in path component.

Summary

Types

error()

Error tuple indicating the type of error. Possible values of the second component

uri_map()

Map holding the main components of a URI.

uri_string()

List of unicode codepoints, a UTF-8 encoded binary, or a mix of the two, representing an RFC 3986 compliant URI (percent-encoded form). A URI is a sequence of characters from a very limited set: the letters of the basic Latin alphabet, digits, and a few special characters.

Functions

allowed_characters()

This is a utility function meant to be used in the shell for printing the allowed characters in each major URI component, and also in the most important characters sets.

compose_query(QueryList)

Composes a form-urlencoded QueryString based on a QueryList, a list of non-percent-encoded key-value pairs.

compose_query(QueryList, Options)

Same as compose_query/1 but with an additional Options parameter, that controls the encoding ("charset") used by the encoding algorithm.

dissect_query(QueryString)

Dissects an urlencoded QueryString and returns a QueryList, a list of non-percent-encoded key-value pairs.

normalize(URI)

Transforms an URI into a normalized form using Syntax-Based Normalization as defined by RFC 3986.

normalize(URI, Options)

Same as normalize/1 but with an additional Options parameter, that controls whether the normalized URI shall be returned as an uri_map().

parse(URIString)

Parses an RFC 3986 compliant uri_string/0 into a uri_map/0, that holds the parsed components of the URI. If parsing fails, an error tuple is returned.

percent_decode(URI)

Decodes all percent-encoded triplets in the input that can be both a uri_string/0 and a uri_map/0.

quote(Data)

Replaces characters out of unreserved set with their percent encoded equivalents.

quote(Data, Safe)

Same as quote/1, but Safe allows user to provide a list of characters to be protected from encoding.

recompose(URIMap)

Creates an RFC 3986 compliant URIString (percent-encoded), based on the components of URIMap. If the URIMap is invalid, an error tuple is returned.

resolve(RefURI, BaseURI)

Convert a RefURI reference that might be relative to a given base URI into the parsed components of the reference's target, which can then be recomposed to form the target URI.

resolve(RefURI, BaseURI, Options)

Same as resolve/2 but with an additional Options parameter, that controls whether the target URI shall be returned as an uri_map(). There is one supported option: return_map.

transcode(URIString, Options)

Transcodes an RFC 3986 compliant URIString, where Options is a list of tagged tuples, specifying the inbound (in_encoding) and outbound (out_encoding) encodings.

unquote(QuotedData)

Percent decode characters.

Types

error()

(since OTP 21.0)

-type error() :: {error, atom(), term()}.

Error tuple indicating the type of error. Possible values of the second component:

invalid_character

invalid_encoding

invalid_input

invalid_map

invalid_percent_encoding

invalid_scheme

invalid_uri

invalid_utf8

missing_value

The third component is a term providing additional information about the cause of the error.

uri_map()

(since OTP 21.0)

-type uri_map() ::
          #{fragment => unicode:chardata(),
            host => unicode:chardata(),
            path => unicode:chardata(),
            port => non_neg_integer() | undefined,
            query => unicode:chardata(),
            scheme => unicode:chardata(),
            userinfo => unicode:chardata()}.

Map holding the main components of a URI.

uri_string()

(since OTP 21.0)

-type uri_string() :: iodata().

List of unicode codepoints, a UTF-8 encoded binary, or a mix of the two, representing an RFC 3986 compliant URI (percent-encoded form). A URI is a sequence of characters from a very limited set: the letters of the basic Latin alphabet, digits, and a few special characters.

Functions

allowed_characters()

(since OTP 23.2)

-spec allowed_characters() -> [{atom(), list()}].

This is a utility function meant to be used in the shell for printing the allowed characters in each major URI component, and also in the most important characters sets.

Note that this function does not replace the ABNF rules defined by the standards, these character sets are derived directly from those aformentioned rules. For more information see the Uniform Resource Identifiers chapter in stdlib's Users Guide.

compose_query(QueryList)

(since OTP 21.0)

-spec compose_query(QueryList) -> QueryString
                       when
                           QueryList :: [{unicode:chardata(), unicode:chardata() | true}],
                           QueryString :: uri_string() | error().

Composes a form-urlencoded QueryString based on a QueryList, a list of non-percent-encoded key-value pairs.

Form-urlencoding is defined in section 4.10.21.6 of the HTML 5.2 specification and in section 4.10.22.6 of the HTML 5.0 specification for non-UTF-8 encodings.

See also the opposite operation dissect_query/1.

Example:

1> uri_string:compose_query([{"foo bar","1"},{"city","örebro"}]).
"foo+bar=1&city=%C3%B6rebro"
2> uri_string:compose_query([{<<"foo bar">>,<<"1">>},
2> {<<"city">>,<<"örebro"/utf8>>}]).
<<"foo+bar=1&city=%C3%B6rebro">>

compose_query(QueryList, Options)

(since OTP 21.0)

-spec compose_query(QueryList, Options) -> QueryString
                       when
                           QueryList :: [{unicode:chardata(), unicode:chardata() | true}],
                           Options :: [{encoding, atom()}],
                           QueryString :: uri_string() | error().

Same as compose_query/1 but with an additional Options parameter, that controls the encoding ("charset") used by the encoding algorithm.

There are two supported encodings: utf8 (or unicode) and latin1.

Each character in the entry's name and value that cannot be expressed using the selected character encoding, is replaced by a string consisting of a U+0026 AMPERSAND character (&), a "#" (U+0023) character, one or more ASCII digits representing the Unicode code point of the character in base ten, and finally a ";" (U+003B) character.

Bytes that are out of the range 0x2A, 0x2D, 0x2E, 0x30 to 0x39, 0x41 to 0x5A, 0x5F, 0x61 to 0x7A, are percent-encoded (U+0025 PERCENT SIGN character (%) followed by uppercase ASCII hex digits representing the hexadecimal value of the byte).

See also the opposite operation dissect_query/1.

Example:

1> uri_string:compose_query([{"foo bar","1"},{"city","örebro"}],
1> [{encoding, latin1}]).
"foo+bar=1&city=%F6rebro"
2> uri_string:compose_query([{<<"foo bar">>,<<"1">>},
2> {<<"city">>,<<"東京"/utf8>>}], [{encoding, latin1}]).
<<"foo+bar=1&city=%26%2326481%3B%26%2320140%3B">>

dissect_query(QueryString)

(since OTP 21.0)

-spec dissect_query(QueryString) -> QueryList
                       when
                           QueryString :: uri_string(),
                           QueryList :: [{unicode:chardata(), unicode:chardata() | true}] | error().

Dissects an urlencoded QueryString and returns a QueryList, a list of non-percent-encoded key-value pairs.

Form-urlencoding is defined in section 4.10.21.6 of the HTML 5.2 specification and in section 4.10.22.6 of the HTML 5.0 specification for non-UTF-8 encodings.

See also the opposite operation compose_query/1.

Example:

1> uri_string:dissect_query("foo+bar=1&city=%C3%B6rebro").
[{"foo bar","1"},{"city","örebro"}]
2> uri_string:dissect_query(<<"foo+bar=1&city=%26%2326481%3B%26%2320140%3B">>).
[{<<"foo bar">>,<<"1">>},
 {<<"city">>,<<230,157,177,228,186,172>>}]

normalize(URI)

(since OTP 21.0)

-spec normalize(URI) -> NormalizedURI
                   when URI :: uri_string() | uri_map(), NormalizedURI :: uri_string() | error().

Transforms an URI into a normalized form using Syntax-Based Normalization as defined by RFC 3986.

This function implements case normalization, percent-encoding normalization, path segment normalization and scheme based normalization for HTTP(S) with basic support for FTP, SSH, SFTP and TFTP.

Example:

1> uri_string:normalize("/a/b/c/./../../g").
"/a/g"
2> uri_string:normalize(<<"mid/content=5/../6">>).
<<"mid/6">>
3> uri_string:normalize("http://localhost:80").
"http://localhost/"
4> uri_string:normalize(#{scheme => "http",port => 80,path => "/a/b/c/./../../g",
4> host => "localhost-örebro"}).
"http://localhost-%C3%B6rebro/a/g"

normalize(URI, Options)

(since OTP 21.0)

-spec normalize(URI, Options) -> NormalizedURI
                   when
                       URI :: uri_string() | uri_map(),
                       Options :: [return_map],
                       NormalizedURI :: uri_string() | uri_map() | error().

Same as normalize/1 but with an additional Options parameter, that controls whether the normalized URI shall be returned as an uri_map().

There is one supported option: return_map.

Example:

1> uri_string:normalize("/a/b/c/./../../g", [return_map]).
#{path => "/a/g"}
2> uri_string:normalize(<<"mid/content=5/../6">>, [return_map]).
#{path => <<"mid/6">>}
3> uri_string:normalize("http://localhost:80", [return_map]).
#{scheme => "http",path => "/",host => "localhost"}
4> uri_string:normalize(#{scheme => "http",port => 80,path => "/a/b/c/./../../g",
4> host => "localhost-örebro"}, [return_map]).
#{scheme => "http",path => "/a/g",host => "localhost-örebro"}

parse(URIString)

(since OTP 21.0)

-spec parse(URIString) -> URIMap when URIString :: uri_string(), URIMap :: uri_map() | error().

Parses an RFC 3986 compliant uri_string/0 into a uri_map/0, that holds the parsed components of the URI. If parsing fails, an error tuple is returned.

See also the opposite operation recompose/1.

Example:

1> uri_string:parse("foo://user@example.com:8042/over/there?name=ferret#nose").
#{fragment => "nose",host => "example.com",
  path => "/over/there",port => 8042,query => "name=ferret",
  scheme => foo,userinfo => "user"}
2> uri_string:parse(<<"foo://user@example.com:8042/over/there?name=ferret">>).
#{host => <<"example.com">>,path => <<"/over/there">>,
  port => 8042,query => <<"name=ferret">>,scheme => <<"foo">>,
  userinfo => <<"user">>}

percent_decode(URI)

(since OTP 23.2)

-spec percent_decode(URI) -> Result
                        when
                            URI :: uri_string() | uri_map(),
                            Result ::
                                uri_string() |
                                uri_map() |
                                {error, {invalid, {atom(), {term(), term()}}}} |
                                error().

Decodes all percent-encoded triplets in the input that can be both a uri_string/0 and a uri_map/0.

Note, that this function performs raw decoding and it shall be used on already parsed URI components. Applying this function directly on a standard URI can effectively change it.

If the input encoding is not UTF-8, an error tuple is returned.

Example:

1> uri_string:percent_decode(#{host => "localhost-%C3%B6rebro",path => [],
1> scheme => "http"}).
#{host => "localhost-örebro",path => [],scheme => "http"}
2> uri_string:percent_decode(<<"%C3%B6rebro">>).
<<"örebro"/utf8>>

Warning

Using uri_string:percent_decode/1 directly on a URI is not safe. This example shows, that after each consecutive application of the function the resulting URI will be changed. None of these URIs refer to the same resource.

3> uri_string:percent_decode(<<"http://local%252Fhost/path">>).
<<"http://local%2Fhost/path">>
4> uri_string:percent_decode(<<"http://local%2Fhost/path">>).
<<"http://local/host/path">>

quote(Data)

(since OTP 25.0)

-spec quote(Data) -> QuotedData when Data :: unicode:chardata(), QuotedData :: unicode:chardata().

Replaces characters out of unreserved set with their percent encoded equivalents.

Unreserved characters defined in RFC 3986 are not quoted.

Example:

1> uri_string:quote("SomeId/04").
"SomeId%2F04"
2> uri_string:quote(<<"SomeId/04">>).
<<"SomeId%2F04">>

Warning

Function is not aware about any URI component context and should not be used on whole URI. If applied more than once on the same data, might produce unexpected results.

quote(Data, Safe)

(since OTP 25.0)

-spec quote(Data, Safe) -> QuotedData
               when Data :: unicode:chardata(), Safe :: string(), QuotedData :: unicode:chardata().

Same as quote/1, but Safe allows user to provide a list of characters to be protected from encoding.

Example:

1> uri_string:quote("SomeId/04", "/").
"SomeId/04"
2> uri_string:quote(<<"SomeId/04">>, "/").
<<"SomeId/04">>

Warning

Function is not aware about any URI component context and should not be used on whole URI. If applied more than once on the same data, might produce unexpected results.

recompose(URIMap)

(since OTP 21.0)

-spec recompose(URIMap) -> URIString when URIMap :: uri_map(), URIString :: uri_string() | error().

Creates an RFC 3986 compliant URIString (percent-encoded), based on the components of URIMap. If the URIMap is invalid, an error tuple is returned.

See also the opposite operation parse/1.

Example:

1> URIMap = #{fragment => "nose", host => "example.com", path => "/over/there",
1> port => 8042, query => "name=ferret", scheme => "foo", userinfo => "user"}.
#{fragment => "nose",host => "example.com",
  path => "/over/there",port => 8042,query => "name=ferret",
  scheme => "foo",userinfo => "user"}

2> uri_string:recompose(URIMap).
"foo://example.com:8042/over/there?name=ferret#nose"

resolve(RefURI, BaseURI)

(since OTP 22.3)

-spec resolve(RefURI, BaseURI) -> TargetURI
                 when
                     RefURI :: uri_string() | uri_map(),
                     BaseURI :: uri_string() | uri_map(),
                     TargetURI :: uri_string() | error().

Convert a RefURI reference that might be relative to a given base URI into the parsed components of the reference's target, which can then be recomposed to form the target URI.

Example:

1> uri_string:resolve("/abs/ol/ute", "http://localhost/a/b/c?q").
"http://localhost/abs/ol/ute"
2> uri_string:resolve("../relative", "http://localhost/a/b/c?q").
"http://localhost/a/relative"
3> uri_string:resolve("http://localhost/full", "http://localhost/a/b/c?q").
"http://localhost/full"
4> uri_string:resolve(#{path => "path", query => "xyz"}, "http://localhost/a/b/c?q").
"http://localhost/a/b/path?xyz"

resolve(RefURI, BaseURI, Options)

(since OTP 22.3)

-spec resolve(RefURI, BaseURI, Options) -> TargetURI
                 when
                     RefURI :: uri_string() | uri_map(),
                     BaseURI :: uri_string() | uri_map(),
                     Options :: [return_map],
                     TargetURI :: uri_string() | uri_map() | error().

Same as resolve/2 but with an additional Options parameter, that controls whether the target URI shall be returned as an uri_map(). There is one supported option: return_map.

Example:

1> uri_string:resolve("/abs/ol/ute", "http://localhost/a/b/c?q", [return_map]).
#{host => "localhost",path => "/abs/ol/ute",scheme => "http"}
2> uri_string:resolve(#{path => "/abs/ol/ute"}, #{scheme => "http",
2> host => "localhost", path => "/a/b/c?q"}, [return_map]).
#{host => "localhost",path => "/abs/ol/ute",scheme => "http"}

transcode(URIString, Options)

(since OTP 21.0)

-spec transcode(URIString, Options) -> Result
                   when
                       URIString :: uri_string(),
                       Options ::
                           [{in_encoding, unicode:encoding()} | {out_encoding, unicode:encoding()}],
                       Result :: uri_string() | error().

Transcodes an RFC 3986 compliant URIString, where Options is a list of tagged tuples, specifying the inbound (in_encoding) and outbound (out_encoding) encodings.

in_encoding and out_encoding specifies both binary encoding and percent-encoding for the input and output data. Mixed encoding, where binary encoding is not the same as percent-encoding, is not supported. If an argument is invalid, an error tuple is returned.

Example:

1> uri_string:transcode(<<"foo%00%00%00%F6bar"/utf32>>,
1> [{in_encoding, utf32},{out_encoding, utf8}]).
<<"foo%C3%B6bar"/utf8>>
2> uri_string:transcode("foo%F6bar", [{in_encoding, latin1},
2> {out_encoding, utf8}]).
"foo%C3%B6bar"

unquote(QuotedData)

(since OTP 25.0)

-spec unquote(QuotedData) -> Data when QuotedData :: unicode:chardata(), Data :: unicode:chardata().

Percent decode characters.

Example:

1> uri_string:unquote("SomeId%2F04").
"SomeId/04"
2> uri_string:unquote(<<"SomeId%2F04">>).
<<"SomeId/04">>

Warning

Function is not aware about any URI component context and should not be used on whole URI. If applied more than once on the same data, might produce unexpected results.