[eeps] EEP XXX: Documentation storage and format

[Vlad Dumitrescu]

On Sat, Jan 6, 2018 at 7:56 PM, José Valim <jose.valim@REDACTED> wrote:

> The chunks could also be located in a zip archive, or online, so maybe a
>> {uri, URI} value is more general than a {path, Path} one? For example, the
>> docs for OTP would be much easier to distribute as an archive that doesn't
>> need to be unpacked than one that has to be unpacked at the right place and
>> create hundreds of small files. Similarily, if the docs are not available
>> locally, they could be retrieved from a URL and cached locally.
>>
>
> Regarding URIs, I am not sure they would solve any problem at the chunk
> level because we would need to process the chunk in order to find the URI.
> So the only thing we could possibly put in the URI is the docs itself,
> since the metadata is already in disk, then I wonder why would you have all
> the metadata in disk but not the docs?
>

An URI could be a file:// or zip:// one, the netwoked version is more of an
afterthought (it's made possible without any changes in the data structure).


> It feels supporting URIs and zip files are trying to solve another
> problem, which is the packaging and distribution of the documentation. But
> that is probably best left to auxiliary tooling.
> This reminds of the usage of Erlang archives in Elixir. At the beginning,
> Elixir allowed developers to install archives with a simple command: mix
> archive.install url://to/archive.ez
> The command would download the archive and put it in the filesystem as an
> .ez file. But with time, we started running into limitations of archives.
> You could not access the contents of priv directories easily and we had
> some weird issues on Windows when updating and deleting archives because of
> open file handles. So we chose a simple solution: we unpack the archives
> after installation.
>

I see your point, you are probably right.


> Actually, why restrict this to handling the documentation? If the
>> information about the module is moved as an item in Docs (at the cost of
>> non-uniform key structure), then the top-level is generic and can be used
>> to store all kinds of metadata about code entities (making Docs a map where
>> 'docs' is just one of the keys). I have an use-case (cross-reference
>> information), and maybe even dialyzer could store plt data in this
>> distributed manner (not sure if it would work or help, just guessing).
>>
>
> You can already use BEAM chunks exactly like that. For example, Elixir
> stores documentation today in a chunk named ExDc and we have recently added
> an ExDp chunk that stores deprecated APIs. API for working with those
> chunks can be found in the beam_lib module.
>

What I would be interested in was what you described in the first section
of the EEP, about how to find chunks that aren't inside the BEAM file. It
would be nice if there was a common mechanism for that, so that each tool
doesn't need to reinvent it.

best regards,
Vlad
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/eeps/attachments/20180106/6b7ced68/attachment.htm>