[eeps] EEP 48: Documentation storage and format
José Valim
jose.valim@REDACTED
Wed Jan 10 17:36:56 CET 2018
Thank you Raimo! I have sent a PR based on the discussion in this mailing
list: https://github.com/erlang/eep/pull/5
*José Valimwww.plataformatec.com.br
<http://www.plataformatec.com.br/>Founder and Director of R&D*
On Wed, Jan 10, 2018 at 4:47 PM, Raimo Niskanen <raimo+eeps@REDACTED
> wrote:
> I have added your proposal to the repository as EEP 48:
>
> http://erlang.org/eep/eeps/eep-0048.html
> https://github.com/erlang/eep/blob/master/eeps/eep-0048.md
>
> I changed the quoting of atoms and pathnames to use `code` quoting,
> changed some strange double and single quote characters into the ASCII
> ones, and set José as the main author partly because of him sending in the
> EEP gave me an e-mail address to use obfuscated and partly because he
> declared the others as co-authors.
>
> Thank you for your contribution!
>
> / Raimo Niskanen, EEP editor.
>
>
>
> On Thu, Jan 04, 2018 at 10:59:58PM +0100, José Valim wrote:
> > This EEP proposes an official API documentation storage to be used by
> > by BEAM languages. By standardizing how API documentation is stored,
> > it will be possible to write tools that work across languages.
> >
> > I want to thank Eric and Radek, who co-authored the proposal, as well as
> > Kenneth, Fred, Tristan and Loïc for their feedback.
> >
> > See the attached document.
> >
> > *José Valimwww.plataformatec.com.br
> > <http://www.plataformatec.com.br/>Founder and Director of R&D*
>
> > Author: Eric Bailey, Radek Szymczyszyn, José Valim
> > Status: Draft
> > Type: Standards Track
> > Created: 04-Jan-2018
> > Post-History:
> > ****
> > EEP XX: Documentation storage and format
> > ----
> >
> >
> >
> > Abstract
> > ========
> >
> > This EEP proposes an official API documentation storage to be used by
> > by BEAM languages. By standardizing how API documentation is stored,
> > it will be possible to write tools that work across languages.
> >
> >
> >
> > Rationale
> > =========
> >
> > Currently, different programming languages and libraries running on
> > BEAM devise their own schemas for storing and accessing documentation.
> > For example, Elixir and LFE provide a `h` helper in their shell that
> > can print the documentation of any module:
> >
> > iex> h String
> > A String in Elixir is a UTF-8 encoded binary.
> >
> > However, Elixir is only able to show docs for Elixir modules. LFE is
> > only able to show docs for LFE functions and so on. If documentation
> > is standardized, such features can be easily added to other languages
> > in a way that works consistently across all BEAM languages.
> >
> > Furthermore, each language ends up building their own tools for
> > generating, processing and converting documentation. We hope a unified
> > approach to documentation will improve the compatibility between tools.
> > For instance, an Erlang IDE will be able to show inline documentation
> > for any module and function, regardless if the function is part of OTP,
> > a library or even written in Elixir, LFE or Alpaca.
> >
> > **Note**: in this document, the word “documentation” refers exclusively
> > to the API documentation of modules and functions. Guides, tutorials
> > and others materials are also essential to projects but not the focus
> > of this EEP.
> >
> > **Note**: This EEP is not about documentation format. It is about a
> > mechanism for storing documentation to make it easier to produce other
> > formats. For example, a tool can read the documentation and produce man
> > pages from it.
> >
> >
> >
> > Specification
> > =============
> >
> > This EEP is divided in three parts. The first defines the two
> > places the documentation can be stored, the second defines the shape of
> > the documentation and the third discusses integration with OTP.
> >
> >
> > ## Part 1: the "Docs"storage ##
> >
> > There are two main mechanisms in which BEAM languages store
> documentation:
> > in the filesystem (usually in the /doc directory) and inside ".beam"
> > files.
> >
> > This EEP recognizes both options and aim to support both. To look for
> > documentation for a module name "example", a tool should:
> >
> > 1. Look for "example.beam" in the code path, parse the BEAM file and
> > retrieve the "Docs" chunk
> >
> > 2. If the chunk is not available, it should look for "example.beam"
> > in the code path and find the "doc/chunks/example.chunk" file in
> > the application that defines the "example" module
> >
> > 3. If a ".chunk" file is not available, then documentation is not
> > available
> >
> > The choice of using a chunk or the filesystem is completely up to the
> > language or library. In both cases, the documentation can be added or
> > removed at any moment by stripping the "Docs" chunk or by removing the
> > "doc/chunks" directory.
> >
> > For example, languages like Elixir and LFE attach the "Docs" chunk at
> > compilation time, which can be controlled via a compiler flag. On the
> > other hand, projects like OTP itself will likely generate the
> "doc/chunks"
> > entries on a separate command, completely unrelated from code
> compilation.
> >
> >
> > ## Part 2: the "Docs" format ##
> >
> >
> > In both storages, the documentation is written in the exactly same
> > format: an Erlang term serialized to binary via term_to_binary/1. The
> > term may be optionally compressed when serialized and must follow the
> > type specification below:
> >
> > {docs_v1,
> > Anno :: erl_anno:anno(),
> > Language :: atom(),
> > Format :: mime_type(),
> > ModuleDoc :: binary() | none | hidden,
> > Metadata :: map(),
> > Docs ::
> > [{{Kind, Name, Arity},
> > Anno :: erl_anno:anno(),
> > Signature :: [binary()],
> > Doc :: binary() | none | hidden,
> > Metadata :: map()
> > }]}
> >
> > where in the root tuple we have:
> >
> > * `Anno` - annotation (line, column, file) of the module documentation
> > or of the definition itself (see erl_anno)
> >
> > * `Language` - an atom representing the language, for example:
> > "erlang", "elixir", "lfe", "alpaca", etc
> >
> > * `Format` - the mime type of the documentation, such as
> "text/markdown"
> > (see the FAQ for a discussion on this field)
> >
> > * `ModuleDoc` - a binary with the documentation or the atom "none"
> > in case there is no documentation or the atom "hidden" if
> > documentation has been explicitly disabled for this entry
> >
> > * `Metadata` - a map of atom keys with any term as value. This can be
> > used to add annotations like the "authors" of a module, "deprecated",
> > or anything else a language or documentation tool may find relevant
> >
> > * `Docs` - a list of documentation for other entities (such as
> > functions and types) in the module
> >
> > For each entry in `Docs`, we have:
> >
> > * `{Kind, Name, Arity}` - the kind, name and arity identifying the
> > function, callback, type, etc. The official entities are:
> `function`,
> > `type` and `callback`. Other languages will add their own. For
> > instance, Elixir and LFE may add `macro`
> >
> > * `Anno` - annotation (line, column, file) of the module documentation
> > or of the definition itself (see erl_anno)
> >
> > * `Signature` - the signature of the entity. It is is a list of
> > binaries. Each entry represents a binary in the signature that can
> > be joined with a whitespace or a newline. For example,
> > `["binary_to_atom(Binary, Encoding)", "when is_binary(Binary)"]`
> > may be rendered as as a single line or two lines. It exists
> > exclusively for exhibition purposes
> >
> > * `Doc` - a binary with the documentation or the atom "none"
> > in case there is no documentation or the atom "hidden" if
> > documentation has been explicitly disabled for this entry
> >
> > * `Metadata` - a map of atom keys with any term as value
> >
> > This shared format is the heart of the EEP as it is what effectively
> > allows cross-language collaboration.
> >
> > The `Metadata` field exists to allow languages, tools and libraries to
> > add custom information to each entry. This EEP documents the
> > following metadata keys:
> >
> > * `authors := [binary()]` - a list of authors as binaries
> >
> > * `cross_references := [module() | {module(), {Kind, Name, Arity}}]` -
> > a list of modules or module entries that can be used as cross
> > references when generating documentation
> >
> > * `deprecated := binary()` - when present, it means the current entry
> > is deprecated with a binary that represents the reason for
> > deprecation and a recommendation to replace the deprecated code
> >
> > * `since := binary()` - a binary representing the version such entry
> > was added, such as "1.3.0" or "20.0"
> >
> > Any key may be added to Metadata at any time. Keys that are frequently
> > used by the community can be standardized in future versions.
> >
> >
> > ## Part 3: Integration with OTP ##
> >
> > The last part focuses on integrating the previous parts with OTP docs,
> > tools and workflows. The items below are suggestions and are not
> > necessary for the adoption of this EEP, neither by OTP nor by any other
> > language or library.
> >
> > At this point we should consider changes to OTP such as:
> >
> > * Distributing the `doc/chunks/*.chunk` files as part of OTP and
> > changing the tools that ship with OTP to rely on them. For example,
> > "erl -man lists" could be changed to locate the "lists.chunk" file,
> > parsing the documentation out and then converting it to a man page
> > on the fly. This task may require multiple changes, as OTP stores
> > documentation on XML files as well as directly in the source code.
> > `edoc` itself should likely be augmented with functions that spit
> > out `.chunk` files from the source code
> >
> > * Adding `h(Module)`, `h(Module, Function, Arity)`, and similar to
> > Erlang’s shell to print the documentation of a module or of a
> > given function and arity. This should be able to print docs any
> > other library or language that implements this proposal
> >
> >
> >
> > FAQ
> > ===
> >
> > *Q: Why do we have a Format entry in the documentation?*
> >
> > The main trade-off in the proposal is the documentation format. We have
> > two options:
> >
> > * Allow each language/library/tool to choose their own documentation
> > format
> > * Impose a unified documentation format on all languages
> >
> > A unified format for documentation gives no flexibility to languages and
> > libraries in choosing how documentation is written. As the ecosystem
> > gets more diverse, it will be unlikely to find a format that suits all.
> > For this reason we introduced a Format field that allows each language
> > and library to pick their documentation format. The downside is that,
> > if the Elixir docs are written in Markdown and a language does not know
> > how to format Markdown, then the language will have to choose to either
> > not show the Elixir docs or show them raw (i.e. in Markdown).
> >
> > Erlang is in a privileged position. All languages will be able to
> > support whatever format is chosen for Erlang since all languages run on
> > Erlang and will have direct access to Erlang's tooling.
> >
> > *Q: If I have an Erlang/Elixir/LFE/Alpaca library that uses a custom
> > documentation toolkit, will I also be able to leverage this?*
> >
> > As long as the documentation ends up up in the "Docs" chunk or inside
> > the `doc/chunks` directory, we absolutely do not care how the
> > documentation was originally written. If you use a custom format,
> > you may need to teach your language of choice how to render it though.
> > See the previous question.
> >
> >
> >
> > Copyright
> > =========
> >
> > This document has been placed in the public domain.
> >
> >
> >
> > [EmacsVar]: <> "Local Variables:"
> > [EmacsVar]: <> "mode: indented-text"
> > [EmacsVar]: <> "indent-tabs-mode: nil"
> > [EmacsVar]: <> "sentence-end-double-space: t"
> > [EmacsVar]: <> "fill-column: 70"
> > [EmacsVar]: <> "coding: utf-8"
> > [EmacsVar]: <> "End:"
> > [VimVar]: <> " vim: set fileencoding=utf-8 expandtab shiftwidth=4
> softtabstop=4: "
> > _______________________________________________
> > eeps mailing list
> > eeps@REDACTED
> > http://erlang.org/mailman/listinfo/eeps
>
>
> --
>
> / Raimo Niskanen, Erlang/OTP, Ericsson AB
> _______________________________________________
> eeps mailing list
> eeps@REDACTED
> http://erlang.org/mailman/listinfo/eeps
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/eeps/attachments/20180110/deda1097/attachment.htm>
More information about the eeps
mailing list