[eeps] EEP XXX: Documentation storage and format

[José Valim]

>
> The chunks could also be located in a zip archive, or online, so maybe a
> {uri, URI} value is more general than a {path, Path} one? For example, the
> docs for OTP would be much easier to distribute as an archive that doesn't
> need to be unpacked than one that has to be unpacked at the right place and
> create hundreds of small files. Similarily, if the docs are not available
> locally, they could be retrieved from a URL and cached locally.
>

Regarding URIs, I am not sure they would solve any problem at the chunk
level because we would need to process the chunk in order to find the URI.
So the only thing we could possibly put in the URI is the docs itself,
since the metadata is already in disk, then I wonder why would you have all
the metadata in disk but not the docs?

The other issue with accepting URIs is that it will make consumers of the
chunk very complex. For example, Elixir and LFE will now need to account
that h(some_module) may need to go over the network to get the
documentation to show to users.

It feels supporting URIs and zip files are trying to solve another problem,
which is the packaging and distribution of the documentation. But that is
probably best left to auxiliary tooling.

This reminds of the usage of Erlang archives in Elixir. At the beginning,
Elixir allowed developers to install archives with a simple command: mix
archive.install url://to/archive.ez

The command would download the archive and put it in the filesystem as an
.ez file. But with time, we started running into limitations of archives.
You could not access the contents of priv directories easily and we had
some weird issues on Windows when updating and deleting archives because of
open file handles. So we chose a simple solution: we unpack the archives
after installation.

Now they work as regular applications. priv works fine, the Windows issues
disappeared, and all of those things are handled completely transparently
for users. As of now, we have no use for archives and if it was up for
voting, I would even remove them from OTP.

Actually, why restrict this to handling the documentation? If the
> information about the module is moved as an item in Docs (at the cost of
> non-uniform key structure), then the top-level is generic and can be used
> to store all kinds of metadata about code entities (making Docs a map where
> 'docs' is just one of the keys). I have an use-case (cross-reference
> information), and maybe even dialyzer could store plt data in this
> distributed manner (not sure if it would work or help, just guessing).
>

You can already use BEAM chunks exactly like that. For example, Elixir
stores documentation today in a chunk named ExDc and we have recently added
an ExDp chunk that stores deprecated APIs. API for working with those
chunks can be found in the beam_lib module.

Thanks for the feedback Vlad!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/eeps/attachments/20180106/5c3c6f9f/attachment.htm>