This EEP proposes a structured documentation API for Erlang where the documentation is handled as part of the language parser and included directly in the compiled .beam files, as a replacement for EDoc style comments. Python, Elixir, and Clojure are examples of languages that follow this approach of treating documentation as data rather than code comments.
The main limitation in EDoc today is that the documentation is kept as code comments. This requires an explicit tool to parse said code comments, which complicates access to the docs by IDEs, from the shell, etc. There have been recent improvements in this area by making EDoc compile to EEP 48 but it still requires an explicit step.
Furthermore, the “code comments” approach is more complex implementation wise, as it requires parsing the source code alongside code comments, parsing the code comments, and so on. It is also beneficial to have an explicit distinction between documentation and code comments: they have different requirements and different audiences.
This EEP proposes the addition of two module attributes to Erlang: -doc
and -moduledoc.
As with EEP 48, this proposal pertains exclusively to API references and their documentation. It doesn’t cover guides, tutorials, and other documentation formats.
This EEP proposes two new attributes: -doc and -moduledoc. They could
be used as follows:
-module(base64).
-moduledoc "
Convenience functions for encoding and decoding from base64.
".
-doc "
Encodes the given binary to base64.
".
-spec encode(binary()) -> binary().
encode(Binary) ->
% ....
-doc "
Decodes the given binary from base64.
".
-spec decode(binary()) -> {ok, binary()} | error.
encode(Binary) ->
% ....
The new -moduledoc attribute can be listed anywhere and it will contain
the documentation for the given module. The -doc attribute must be
listed anywhere before a function, type attribute or callback attribute and it will
contain the documentation for the following function, type or callback. For instance,
the example below:
-doc "Example".
-spec example() -> ok.
example() -> ok.
is equivalent to:
-spec example() -> ok.
-doc "Example".
example() -> ok.
Listing multiple -doc attributes with string values for the same
function should warn or error accordingly, unless the documentation
is being set to hidden. For example, this is valid:
-doc "Example".
-doc hidden.
example() -> ok.
But this should warn/error:
-doc "Example".
-doc "Updated example".
example() -> ok.
This as well:
-doc "Example".
example(one) -> 1;
-doc "Updated example".
example(two) -> 2;
The module attribute must either be a string OR the atom false.
Marking a module as hidden means it won’t be part of the doc.
For example, imagine the base64 module above delegates some of
its logic to a private base64_impl module:
-module(base64_impl).
-moduledoc false.
Note a module may be hidden but individual functions can still be documented:
-module(base64_impl).
-moduledoc false.
-doc "
Some comments as if it was public.
".
decode64(Binary) ->
% ...
According to EEP 48, this is intentional. For example, base64_impl
should be private for users of the base64 functionality, but a
developer working directly on the base64 may still want to access
the docs for base64_impl functions directly from their IDE. Each
documentation tool should honor hidden accordingly. If no -doc
is provided, it defaults to none according to EEP 48.
The -doc attribute accepts the false atom too.
Some developers prefer to not place the documentation alongside the
source code. For such cases, -doc and -moduledoc may also provide
a {file, Path}, where Path is a relative path from file containing the
doc attribute:
-moduledoc({file, "../doc/src/manual/my_module.asciidoc"}).
-doc({file, "../doc/src/manual/my_module.my_function.asciidoc"}).
The file will be read by the compiler and embedded into the chunk at compilation time.
The -doc attribute can also be used to document types and callbacks.
The -doc attribute can be used for private function as well, so that
tools and IDEs can provide docs if the user wants them to. However,
the private function should not end up in the EEP 48 doc chunk.
In general private types are handled just as private functions, however there are times when types are used for documentation and code-sharing purposes within a module, but the user does not want to export it for general use. An example of this are all the various option types in the ssl module.
Therefore any private type that is referred to by a public function
specification or type will also be included in the documentation chunks.
Such types will have the exported metadata key set to false.
The new module attributes must also support documentation metadata by passing a map as argument:
-module(beam64).
-moduledoc "
Convenience functions for encoding and decoding from base64.
".
-moduledoc #{
author => [<<"The Erlang/OTP team">>],
license => <<"Apache 2 License">>,
cross_references => [binary]
}.
If the -moduledoc is called multiple times with a map, the maps will
be merged. This comes with the added benefit that shared metadata can
be moved to a header file:
%% prelude.hrl
-moduledoc #{
authors => [<<"The Erlang/OTP team">>],
license => <<"Apache 2 License">>
}.
which we can then include and augment:
-module(beam64).
-include("prelude.hrl").
-moduledoc "
Convenience functions for encoding and decoding from base64.
".
-moduledoc #{cross_references => [binary]}.
A list of built-in attributes is available on EEP 48.
Compiling a module with the -moduledoc or -doc attributes will generate a
Docs chunk into its .beam file, making the documentation directly accessible in
the shell.
Release tools should also prune the Documentation chunk out of .beam
files by default. Note this is already done by beam_lib:strip_release/1
and beam_lib:strip_files/1.
warn_missing_doc #
The compiler will warn if a function, type or callback that is included
in the EEP 48 doc chunk does not have a -doc attribute set if the
warn_missing_doc flag is passed to the compiler.
The flag can be passed both on the command line as +warn_missing_doc,
or as a -compile(warn_missing_doc) attribute the source code.
One important discussion about documentation is what is the documentation format that the documentation should adopt. Luckily, EEP 48 is agnostic to the format, however one must still be listed.
In order to facilitate multiple documentation formats Erlang/OTP allows the
user to place a format key in the -moduledoc metadata which should specify
the mime type of the format used, as specified by EEP 48. The default format
will be text/markdown.
-moduledoc(#{ format => "text/edoc" }).
A direct consequence of making the documentation more structured and accessible is that Erlang can include doctests, which is the ability to run and validate the examples in your documentation. For example, someone could write this:
-doc """
Encodes the given binary to base64.
1> base64:encode("hello").
<<"aGVsbG8=">>
""".
-spec encode(binary()) -> binary().
encode(Binary) ->
% ....
And then in your test suite:
doctests(_Config) ->
ct_doctest:run(base64).
The doctest attribute will access the documentation entries in the base64 Docs chunk, extract all of the examples, and run them. Of course, while there is nothing stopping doctests from being implemented on top of EDoc today, this EEP makes doctests considerably simpler to implement.
Doctests would benefit from a separate EEP, as there are some extra considerations, as doctesting exceptions, unparseable formats, etc, but it is worth mentioning them given their benefits to users and documentation authors.
erl_docgen? #
If this proposal is to be accepted, what happens with Edoc?
One important aspect of the work proposed by the EEP is to try and
unify the documentation tools in the Erlang/OTP ecosystem. Before
this, there have been multiple tools, EDoc mostly being used by
open source projects, and erl_docgen by the Erlang/OTP project,
and other third party solutions.
In the short-term, EDoc will be updated to be able to generate
html reports from EEP 48 doc chunks containing text/edoc
documentation. The EDoc comment style of specifying documentation
will be deprecated, however, for backwards compatibility the support
to parse the EDoc comments will remain for a long time.
In the long term, the aim is to switch document rendering engine to use ExDoc, which supports Erlang projects either by running it as an escript or via Rebar3 integration. When using ExDoc, the user can choose to either use EDoc syntax or migrate to Markdown.
Similarly, the Erlang/OTP code will be converted to use ExDoc
instead of erl_docgen to generate documentation. All the current
XML documentation files will be converted to use Markdown.
This document is placed in the public domain or under the CC0-1.0-Universal license, whichever is more permissive.