<div dir="ltr">Joe, check out this startup - <a href="https://www.simpla.io/">https://www.simpla.io/</a><div><br></div><div>They allow editable web pages, which creates blobs which get saves directly into a database. No more digging through source files to edit docs, people (with appropriate permission) can edit the page live. It would be very straightforward to write a program to dump from the database to XML/JSON or whatever format you're interested in, making it portable, and fairly trivial to go the other way. (Full disclosure: It's not my startup but they're my mates, we went through the same accelerator).</div><div><br></div><div>Is the raw XML documentation available anywhere? </div><div><br></div><div>Thanks,</div><div>Luke</div></div><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Sep 27, 2016 at 6:03 AM, Joe Armstrong <span dir="ltr"><<a href="mailto:erlang@gmail.com" target="_blank">erlang@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Regarding the documentation - there is a step0 that needs to be done<br>
*before* attacking the document improvement problem.<br>
<br>
The following things are needed (I my opinion)<br>
<br>
1) The number of DTDs in the documentation should be reduced to ONE<br>
there are currently 26 these are in<br>
otp_src_<vsn>/lib/erl_docgen/<wbr>priv/dtd<br>
<br>
2) tags that are used infrequently should be removed and the source XML<br>
corrected. Some tags are virtually *never* used.<br>
<br>
3) All XML files should validate with the new DTD<br>
<br>
These steps need quite a lot of work ...<br>
<br>
4) The Erlang parser should be changed to exactly<br>
reproduce the source.<br>
Right now the parse tree of correct erlang has all the comments<br>
and white space removed. I'd suggest attaching the comments to the<br>
next following token (for example {atom,Line,theAtom} should become<br>
{atom, Line, theAtom, "the preceding comments and white space"}<br>
It should be possible to *exactly* reconstruct the input from the parse<br>
tree.<br>
<br>
<aside> - in the first erlang all the different ways of writing an integer<br>
ended up as the same token. So writing 16#fc was the same as writing the<br>
integer 252 and tokenized as {integer,Line,252} - the tokenizer threw<br>
away the exact input so it was impossible to reconstruct the source<br>
from the token stream. Now it's better the 16#fc is tokenized as<br>
{integer,[{location,{Line,Col}<wbr>},{text,"16$fc}], 252} - but comments<br>
and white space are not<br>
retained in the parse tree.<br>
<br>
Note: that change the parse tree is *not* a simple hack - all tools that<br>
depend upon the parse tree have to be changed.<br>
</aside><br>
<br>
<br>
<br>
<br>
<br>
5) We should decide how to attach "floating" comments in the source.<br>
Does a comment *before* a function apply to the next function or not?<br>
<br>
6) We need some "injection" API to inject code, meta-data, examples<br>
and documentations into a data base.<br>
<br>
For example inject:code("foo.erl") should inject a load of key/value<br>
pairs into a data base, with something like the following keys<br>
<br>
{text, Mod,Func,Arity} => the source code text<br>
{spec, Mod, Func, Arity} => the spec<br>
{doc, Mod,Func,Arity} => the documentation<br>
{examples,Mod,Func,Arity} = [Examples]<br>
<br>
The entities in the database should be sufficient to reconstruct the<br>
original text, and perform various analysis of the functions.<br>
<br>
I think *most* of the problems involved are due to the difficulty of<br>
extracting information from the source files and editing this when it is wrong.<br>
<br>
I'm currently trying to do parts of this.<br>
<br>
A "relatively simple" program should be able to (for a given function)<br>
<br>
- find the exact source text<br>
- find all old versions<br>
- find the specs and types referred to<br>
- find the documentation<br>
- find the test cases<br>
<br>
Doing so involves analysing the erlang sources the XML sources and the<br>
test cases, and involves a deal of guess work.<br>
<br>
All this must be done on a moving target and should not break the<br>
existing system.<br>
<br>
I suspect that the code to accurately manipulate the code and<br>
documentation have been has been written several times in different<br>
projects (for example<br>
the wrangler, and the Eclipse interface) both need to manipulate the source<br>
in various ways.<br>
<span class=""><br>
<br>
<br>
<br>
On Mon, Sep 26, 2016 at 6:41 PM, <<a href="mailto:lloyd@writersglen.com">lloyd@writersglen.com</a>> wrote:<br>
> Joe,<br>
><br>
> You've said so well what I've been trying to harp on.<br>
><br>
> My most recent timesink has been trying to understand xmerl sufficiently well to pull book data out of several different book APIs. Dave Thomas's 2007 tutorial has been a big help, but the black holes in my understanding still significantly impede my progress. So far I've spent maybe 10 to 15 hours trying to scope it out. I can get much of what I need from Amazon's APIs, but I need a redundant source. The Library of Congress API completely eludes me; I get a little further with ISBNdb, but still not far enough.<br>
><br>
> Given discussion on the documentation thread to date, it seems to me that there are four issues at stake:<br>
><br>
> 1) Content deficiencies<br>
> 2) Formatting issues<br>
> 3) Lack of consensus of what we, as a community, want<br>
> 4) How we move forward toward comprehensive improvement of documentation.<br>
><br>
> Lukas Larsson's most recent post makes a good point.<br>
><br>
> Bruce Yinhe tells me in a private post that his group is about to hire one person on a part-time basis to work on documentation improvements.<br>
><br>
> I've lost it in the thread, but as I recall we had some promising interest in documentation improvements from an Erricson employee.<br>
><br>
> It would be great if we could begin to rally around these comments and find some kind of convergence toward progress.<br>
><br>
> My take is to break the large task down into small chunks, bring the intelligence and resources of the community to bear on one specific issue at a time and, and get it done.<br>
<br>
</span>I think many (most) of the problems arise because what we are ultimately<br>
doing is changing the content of a file at some place.<br>
<br>
Fixing a typo/bug involves<br>
<br>
1- finding the appropriate file<br>
2 - changing the file at the appropriate place<br>
3 - updating the file (somehow)<br>
4 - generating the downstream documents that depend upon the file<br>
<br>
All these steps are difficult<br>
<br>
We can imagine a simpler way:<br>
<br>
Suppose a file is a sequence of paragraphs. Each paragraph<br>
has a GUId<br>
<br>
In (say) HTML<br>
<br>
<p guid = "b92a2705-3449-4fb9-8f11-<wbr>fa55f7ead29f"><br>
This is my paragarph ...<br>
</p><br>
<br>
If I want to update the paragraphs I just send a message<br>
<br>
{update,"b92a2705-3449-4fb9-<wbr>8f11-fa55f7ead29f"<br>
"the new content"}<br>
<br>
to some server - this should be checked (manually) and then<br>
if approved used to update everything.<br>
<br>
In other threads I have argued that *everything* should be in a global<br>
database with a huge DHT tracking where things are.<br>
<br>
A key to "changing things" is "naming things" and "finding things"<br>
<br>
yes another (even simpler) alternative is to have a message<br>
<br>
{change,SHA,NewText}<br>
<br>
Meaning "change the paragraph with sha1 checksum <SHA> to <NewText>"<br>
<br>
Implementing this is easy - BUT all paragraphs with the same SHA<br>
would be changed - which might not be what we want.<br>
<br>
I have (incidentally) experimented with this - tagging all paragraphs<br>
with their SHAs and sticking the results in a database.<br>
<br>
<crazy idea><br>
make a server that accepts messages of the form<br>
<br>
{change, <SHA>, <New Text>}<br>
<br>
The server finds a paragraph with sha1 checksum <SHA> and changes<br>
it to <New Text> - it changes the appropriate file in a GIT archive<br>
does all the /add/commit/push magic and the job is done.<br>
<br>
(I think I'll implement this for fun :-)<br>
<br>
</crazy idea><br>
<div class="HOEnZb"><div class="h5"><br>
><br>
> I don't have the technical chops, but I'll gladly work with you or anyone else to address content issues on a module-by-module basis. I can ask Micky-the_Dunce questions and, perhaps, help clarify language. It would be great if you could help clarify intent and application issues.<br>
><br>
> All the best,<br>
><br>
> Lloyd<br>
><br>
><br>
><br>
><br>
><br>
><br>
><br>
><br>
> -----Original Message-----<br>
> From: "Joe Armstrong" <<a href="mailto:erlang@gmail.com">erlang@gmail.com</a>><br>
> Sent: Monday, September 26, 2016 6:42am<br>
> To: "Lukas Larsson" <<a href="mailto:lukas@erlang.org">lukas@erlang.org</a>><br>
> Cc: "Lloyd R. Prentice" <<a href="mailto:lloyd@writersglen.com">lloyd@writersglen.com</a>>, "<a href="mailto:erlang-questions@erlang.org">erlang-questions@erlang.org</a>" <<a href="mailto:erlang-questions@erlang.org">erlang-questions@erlang.org</a>><br>
> Subject: Re: [erlang-questions] Erlang documentation -- a modest proposal<br>
><br>
> I think what I miss most are *examples*<br>
><br>
> I've just been reading the edoc manual pages for a program<br>
> called <XYZ> (name changed to avoid embarrassment)<br>
><br>
> The functions are well documented - they types are well documented<br>
> but I haven't a clue about which ORDER to call the functions<br>
><br>
> Imagine a file system.<br>
><br>
> We *document* the open, read, write, and close functions<br>
> but we don't say you have to open the file before we read it.<br>
> We dont say when we're done we have to close the file.<br>
><br>
> We don't say this because it is *obvious*<br>
><br>
> But for the module glonk, which exports, zizzle, taddle, glonk and plonk<br>
> it is NOT obvious. Yes sure you all know you have to call glonk 3 times<br>
> before calling plonk - but I don't know.<br>
><br>
> Thats why we need examples.<br>
><br>
> Often I search for a tutorial and find a ten line blog posting that<br>
> actually shows me how to use a library - this gets me started.<br>
><br>
> very short unit tests - placed inline are *very* useful<br>
><br>
> for example:<br>
><br>
> "321" = lists:reverse("123")<br>
><br>
> The unit test *are* the examples - what we don't have is software that<br>
> parses the code, parses the documentation, parses the unit test<br>
> and munges all together into a form that is convenient to read.<br>
><br>
> I'm actually trying to write something like this now - hence my wails<br>
> of anguish over css.<br>
><br>
> Wish me luck<br>
><br>
> /Joe<br>
><br>
><br>
><br>
><br>
><br>
><br>
><br>
> On Mon, Sep 26, 2016 at 11:58 AM, Lukas Larsson <<a href="mailto:lukas@erlang.org">lukas@erlang.org</a>> wrote:<br>
>> Hello,<br>
>><br>
>> On Thu, Sep 22, 2016 at 11:56 PM, Lloyd R. Prentice <<a href="mailto:lloyd@writersglen.com">lloyd@writersglen.com</a>><br>
>> wrote:<br>
>>><br>
>>> Hello,<br>
>>><br>
>>> To date, this thread has generated quite a few worthwhile insights and<br>
>>> ideas. My fear is that they will be deep-sixed into the archive. On the<br>
>>> other hand, major revision is a daunting task and unlikely to happen.<br>
>>><br>
>>> But maybe we can focus on specific issues and make iterative headway.<br>
>>><br>
>>> Fewer than half of the functions in the lists library, for instance, have<br>
>>> code examples. Suppose over the span of one week we were collectively focus<br>
>>> on generating at least two code examples for each function in one library.<br>
>>><br>
>>> At the end of the week we could organize the submissions and vote on best<br>
>>> candidates for inclusion in the docs. That done, we can pick another module.<br>
>>><br>
>>> Thus, with not much effort from any one individual, a small posse of<br>
>>> volunteer Erlang wizards could make short work of deficiencies in the docs.<br>
>>><br>
>>> Anyway, it's an idea.<br>
>>><br>
>>> All the best,<br>
>>><br>
>>> LRP<br>
>>><br>
>><br>
>> I think that it is great to see everyone talking about wanting to improve<br>
>> the documentation. The contributions to the Erlang/OTP project that I value<br>
>> that most are documentation changes that make the intention clearer, or<br>
>> explains some corner case somewhere which the docs did not initially<br>
>> mention.<br>
>><br>
>> Unfortunately, once one has figured out how a function works there seems to<br>
>> be very little incentive to make the docs clearer. I would estimate that<br>
>> about every 20th pull request we get is a documentation fix, and more than<br>
>> half of those are fixes of speling misstakes (which are great!).<br>
>><br>
>> I've just come back from about two weeks of vacation and this discussion has<br>
>> resulted in roughly 0 pull requests for changes in the documentation. Would<br>
>> it be possible to steer this discussion into doing something instead of<br>
>> talking about doing something? Yes the technology/layout is not perfect, but<br>
>> as Loïc said, it is the content that matters the most.<br>
>><br>
>> Lukas<br>
>> // my own oppinions<br>
>><br>
>> ______________________________<wbr>_________________<br>
>> erlang-questions mailing list<br>
>> <a href="mailto:erlang-questions@erlang.org">erlang-questions@erlang.org</a><br>
>> <a href="http://erlang.org/mailman/listinfo/erlang-questions" rel="noreferrer" target="_blank">http://erlang.org/mailman/<wbr>listinfo/erlang-questions</a><br>
>><br>
><br>
><br>
______________________________<wbr>_________________<br>
erlang-questions mailing list<br>
<a href="mailto:erlang-questions@erlang.org">erlang-questions@erlang.org</a><br>
<a href="http://erlang.org/mailman/listinfo/erlang-questions" rel="noreferrer" target="_blank">http://erlang.org/mailman/<wbr>listinfo/erlang-questions</a><br>
</div></div></blockquote></div><br></div>