[eeps] EEP 48: Documentation storage and format
ScriptCulture
erlang@REDACTED
Thu Jan 11 23:36:26 CET 2018
-José—Eric—Radek—
This is fantastic news...especially with Code Beam coming up shortly.
Attached is a typeset single-page printout of the root tuple & description.
I find those easier to meditate on.
It's attached.
Cheers,
Robert C.
On 1/10/18 11:36 AM, José Valim wrote:
> Thank you Raimo! I have sent a PR based on the discussion in this
> mailing list: https://github.com/erlang/eep/pull/5
>
>
>
>
>
> *
> *José Valim*
> www.plataformatec.com.br <http://www.plataformatec.com.br/>
> Founder and *
> Director of R&D
> *
> *
>
> On Wed, Jan 10, 2018 at 4:47 PM, Raimo Niskanen
> <raimo+eeps@REDACTED <mailto:raimo+eeps@REDACTED>> wrote:
>
> I have added your proposal to the repository as EEP 48:
>
> http://erlang.org/eep/eeps/eep-0048.html
> <http://erlang.org/eep/eeps/eep-0048.html>
> https://github.com/erlang/eep/blob/master/eeps/eep-0048.md
> <https://github.com/erlang/eep/blob/master/eeps/eep-0048.md>
>
> I changed the quoting of atoms and pathnames to use `code` quoting,
> changed some strange double and single quote characters into the ASCII
> ones, and set José as the main author partly because of him
> sending in the
> EEP gave me an e-mail address to use obfuscated and partly because he
> declared the others as co-authors.
>
> Thank you for your contribution!
>
> / Raimo Niskanen, EEP editor.
>
>
>
> On Thu, Jan 04, 2018 at 10:59:58PM +0100, José Valim wrote:
> > This EEP proposes an official API documentation storage to be
> used by
> > by BEAM languages. By standardizing how API documentation is
> stored,
> > it will be possible to write tools that work across languages.
> >
> > I want to thank Eric and Radek, who co-authored the proposal, as
> well as
> > Kenneth, Fred, Tristan and Loïc for their feedback.
> >
> > See the attached document.
> >
> > *José Valimwww.plataformatec.com.br
> <http://Valimwww.plataformatec.com.br>
> > <http://www.plataformatec.com.br/
> <http://www.plataformatec.com.br/>>Founder and Director of R&D*
>
> > Author: Eric Bailey, Radek Szymczyszyn, José Valim
> > Status: Draft
> > Type: Standards Track
> > Created: 04-Jan-2018
> > Post-History:
> > ****
> > EEP XX: Documentation storage and format
> > ----
> >
> >
> >
> > Abstract
> > ========
> >
> > This EEP proposes an official API documentation storage to be
> used by
> > by BEAM languages. By standardizing how API documentation is
> stored,
> > it will be possible to write tools that work across languages.
> >
> >
> >
> > Rationale
> > =========
> >
> > Currently, different programming languages and libraries running on
> > BEAM devise their own schemas for storing and accessing
> documentation.
> > For example, Elixir and LFE provide a `h` helper in their shell that
> > can print the documentation of any module:
> >
> > iex> h String
> > A String in Elixir is a UTF-8 encoded binary.
> >
> > However, Elixir is only able to show docs for Elixir modules.
> LFE is
> > only able to show docs for LFE functions and so on. If
> documentation
> > is standardized, such features can be easily added to other
> languages
> > in a way that works consistently across all BEAM languages.
> >
> > Furthermore, each language ends up building their own tools for
> > generating, processing and converting documentation. We hope a
> unified
> > approach to documentation will improve the compatibility between
> tools.
> > For instance, an Erlang IDE will be able to show inline
> documentation
> > for any module and function, regardless if the function is part
> of OTP,
> > a library or even written in Elixir, LFE or Alpaca.
> >
> > **Note**: in this document, the word “documentation” refers
> exclusively
> > to the API documentation of modules and functions. Guides, tutorials
> > and others materials are also essential to projects but not the
> focus
> > of this EEP.
> >
> > **Note**: This EEP is not about documentation format. It is about a
> > mechanism for storing documentation to make it easier to produce
> other
> > formats. For example, a tool can read the documentation and
> produce man
> > pages from it.
> >
> >
> >
> > Specification
> > =============
> >
> > This EEP is divided in three parts. The first defines the two
> > places the documentation can be stored, the second defines the
> shape of
> > the documentation and the third discusses integration with OTP.
> >
> >
> > ## Part 1: the "Docs"storage ##
> >
> > There are two main mechanisms in which BEAM languages store
> documentation:
> > in the filesystem (usually in the /doc directory) and inside ".beam"
> > files.
> >
> > This EEP recognizes both options and aim to support both. To
> look for
> > documentation for a module name "example", a tool should:
> >
> > 1. Look for "example.beam" in the code path, parse the BEAM
> file and
> > retrieve the "Docs" chunk
> >
> > 2. If the chunk is not available, it should look for
> "example.beam"
> > in the code path and find the "doc/chunks/example.chunk"
> file in
> > the application that defines the "example" module
> >
> > 3. If a ".chunk" file is not available, then documentation is not
> > available
> >
> > The choice of using a chunk or the filesystem is completely up
> to the
> > language or library. In both cases, the documentation can be
> added or
> > removed at any moment by stripping the "Docs" chunk or by
> removing the
> > "doc/chunks" directory.
> >
> > For example, languages like Elixir and LFE attach the "Docs"
> chunk at
> > compilation time, which can be controlled via a compiler flag.
> On the
> > other hand, projects like OTP itself will likely generate the
> "doc/chunks"
> > entries on a separate command, completely unrelated from code
> compilation.
> >
> >
> > ## Part 2: the "Docs" format ##
> >
> >
> > In both storages, the documentation is written in the exactly same
> > format: an Erlang term serialized to binary via
> term_to_binary/1. The
> > term may be optionally compressed when serialized and must
> follow the
> > type specification below:
> >
> > {docs_v1,
> > Anno :: erl_anno:anno(),
> > Language :: atom(),
> > Format :: mime_type(),
> > ModuleDoc :: binary() | none | hidden,
> > Metadata :: map(),
> > Docs ::
> > [{{Kind, Name, Arity},
> > Anno :: erl_anno:anno(),
> > Signature :: [binary()],
> > Doc :: binary() | none | hidden,
> > Metadata :: map()
> > }]}
> >
> > where in the root tuple we have:
> >
> > * `Anno` - annotation (line, column, file) of the module
> documentation
> > or of the definition itself (see erl_anno)
> >
> > * `Language` - an atom representing the language, for example:
> > "erlang", "elixir", "lfe", "alpaca", etc
> >
> > * `Format` - the mime type of the documentation, such as
> "text/markdown"
> > (see the FAQ for a discussion on this field)
> >
> > * `ModuleDoc` - a binary with the documentation or the atom "none"
> > in case there is no documentation or the atom "hidden" if
> > documentation has been explicitly disabled for this entry
> >
> > * `Metadata` - a map of atom keys with any term as value.
> This can be
> > used to add annotations like the "authors" of a module,
> "deprecated",
> > or anything else a language or documentation tool may find
> relevant
> >
> > * `Docs` - a list of documentation for other entities (such as
> > functions and types) in the module
> >
> > For each entry in `Docs`, we have:
> >
> > * `{Kind, Name, Arity}` - the kind, name and arity identifying the
> > function, callback, type, etc. The official entities are:
> `function`,
> > `type` and `callback`. Other languages will add their own. For
> > instance, Elixir and LFE may add `macro`
> >
> > * `Anno` - annotation (line, column, file) of the module
> documentation
> > or of the definition itself (see erl_anno)
> >
> > * `Signature` - the signature of the entity. It is is a list of
> > binaries. Each entry represents a binary in the signature
> that can
> > be joined with a whitespace or a newline. For example,
> > `["binary_to_atom(Binary, Encoding)", "when is_binary(Binary)"]`
> > may be rendered as as a single line or two lines. It exists
> > exclusively for exhibition purposes
> >
> > * `Doc` - a binary with the documentation or the atom "none"
> > in case there is no documentation or the atom "hidden" if
> > documentation has been explicitly disabled for this entry
> >
> > * `Metadata` - a map of atom keys with any term as value
> >
> > This shared format is the heart of the EEP as it is what effectively
> > allows cross-language collaboration.
> >
> > The `Metadata` field exists to allow languages, tools and
> libraries to
> > add custom information to each entry. This EEP documents the
> > following metadata keys:
> >
> > * `authors := [binary()]` - a list of authors as binaries
> >
> > * `cross_references := [module() | {module(), {Kind, Name,
> Arity}}]` -
> > a list of modules or module entries that can be used as cross
> > references when generating documentation
> >
> > * `deprecated := binary()` - when present, it means the
> current entry
> > is deprecated with a binary that represents the reason for
> > deprecation and a recommendation to replace the deprecated code
> >
> > * `since := binary()` - a binary representing the version such
> entry
> > was added, such as "1.3.0" or "20.0"
> >
> > Any key may be added to Metadata at any time. Keys that are
> frequently
> > used by the community can be standardized in future versions.
> >
> >
> > ## Part 3: Integration with OTP ##
> >
> > The last part focuses on integrating the previous parts with OTP
> docs,
> > tools and workflows. The items below are suggestions and are not
> > necessary for the adoption of this EEP, neither by OTP nor by
> any other
> > language or library.
> >
> > At this point we should consider changes to OTP such as:
> >
> > * Distributing the `doc/chunks/*.chunk` files as part of OTP and
> > changing the tools that ship with OTP to rely on them. For
> example,
> > "erl -man lists" could be changed to locate the
> "lists.chunk" file,
> > parsing the documentation out and then converting it to a
> man page
> > on the fly. This task may require multiple changes, as OTP
> stores
> > documentation on XML files as well as directly in the source
> code.
> > `edoc` itself should likely be augmented with functions that
> spit
> > out `.chunk` files from the source code
> >
> > * Adding `h(Module)`, `h(Module, Function, Arity)`, and similar to
> > Erlang’s shell to print the documentation of a module or of a
> > given function and arity. This should be able to print docs any
> > other library or language that implements this proposal
> >
> >
> >
> > FAQ
> > ===
> >
> > *Q: Why do we have a Format entry in the documentation?*
> >
> > The main trade-off in the proposal is the documentation format.
> We have
> > two options:
> >
> > * Allow each language/library/tool to choose their own
> documentation
> > format
> > * Impose a unified documentation format on all languages
> >
> > A unified format for documentation gives no flexibility to
> languages and
> > libraries in choosing how documentation is written. As the
> ecosystem
> > gets more diverse, it will be unlikely to find a format that
> suits all.
> > For this reason we introduced a Format field that allows each
> language
> > and library to pick their documentation format. The downside is
> that,
> > if the Elixir docs are written in Markdown and a language does
> not know
> > how to format Markdown, then the language will have to choose to
> either
> > not show the Elixir docs or show them raw (i.e. in Markdown).
> >
> > Erlang is in a privileged position. All languages will be able to
> > support whatever format is chosen for Erlang since all languages
> run on
> > Erlang and will have direct access to Erlang's tooling.
> >
> > *Q: If I have an Erlang/Elixir/LFE/Alpaca library that uses a custom
> > documentation toolkit, will I also be able to leverage this?*
> >
> > As long as the documentation ends up up in the "Docs" chunk or
> inside
> > the `doc/chunks` directory, we absolutely do not care how the
> > documentation was originally written. If you use a custom format,
> > you may need to teach your language of choice how to render it
> though.
> > See the previous question.
> >
> >
> >
> > Copyright
> > =========
> >
> > This document has been placed in the public domain.
> >
> >
> >
> > [EmacsVar]: <> "Local Variables:"
> > [EmacsVar]: <> "mode: indented-text"
> > [EmacsVar]: <> "indent-tabs-mode: nil"
> > [EmacsVar]: <> "sentence-end-double-space: t"
> > [EmacsVar]: <> "fill-column: 70"
> > [EmacsVar]: <> "coding: utf-8"
> > [EmacsVar]: <> "End:"
> > [VimVar]: <> " vim: set fileencoding=utf-8 expandtab
> shiftwidth=4 softtabstop=4: "
> > _______________________________________________
> > eeps mailing list
> > eeps@REDACTED <mailto:eeps@REDACTED>
> > http://erlang.org/mailman/listinfo/eeps
> <http://erlang.org/mailman/listinfo/eeps>
>
>
> --
>
> / Raimo Niskanen, Erlang/OTP, Ericsson AB
> _______________________________________________
> eeps mailing list
> eeps@REDACTED <mailto:eeps@REDACTED>
> http://erlang.org/mailman/listinfo/eeps
> <http://erlang.org/mailman/listinfo/eeps>
>
>
>
>
> _______________________________________________
> eeps mailing list
> eeps@REDACTED
> http://erlang.org/mailman/listinfo/eeps
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/eeps/attachments/20180111/852443a7/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: eep_48_printout_jan_10_225k.png
Type: image/png
Size: 231465 bytes
Desc: not available
URL: <http://erlang.org/pipermail/eeps/attachments/20180111/852443a7/attachment.png>
More information about the eeps
mailing list