discuss: Thread: History of LDP

I think it's about time to update the history of LDP and perhaps be more
honest about it.  I was sent to the Montreal expo in 2000 and almost
everyone visiting the LDP booth there complained about our documentation
being out-of-date.  After that, the problem only got worse except that we
started reviewing docs more which improved the quality of some docs.

My view is that we made a big mistake by going with DocBook.  Part of this
was my fault for being partially ignorant of the situation along with our
leader, Guylhen, also not understanding it and thinking that one could
simplify DocBook by using only a subset of it.  I was somewhat of a coward
for not being more strongly opposed to DocBook.

Do we have good archives of the "discuss" list that one can search with a
search engine?  I kinda doubt it.  I don't think that Google has the
archive in it's index either since I've done some test searches and come
up with nothing.  But I've saved most of my outgoing emails so I have some
info to use.

Publishing ones work on ones own website is not always a good idea, unless
there is someone available to take over the doc if the author is no longer
able to maintain it.  And this happens a lot.  People get busy with other
activities and sometimes even die.

One project for ldp is to find good linux docs on personal websites and
suggest they submit them to LDP.  The history should include not only what
LDP did but what it failed to do.  One was to create a content management
system which would allow authors to request that someone else maintain
their doc.

One question I have is why around 2000 a number of women got involved with
LDP and then bowed out.  Did Joy Goodreau, who worked for IBM, have
anything to do with this.  She tried to improve our documentation, partly
by writing stuff on how to write and formulated a bunch of rules, etc. but
most people who wrote for LDP don't want to spend much time studying this
sort of stuff.

Another was the wiki problem.  People would be more likely to edit docs if
one could just do it without registering.  But it takes a lot of effort to
stop spam, including blocking ranges of leased urls that generate spam.
If ldp couldn't find the people to deal with this, perhaps ldp shouldn't
have a wiki.  But ldp could have tried to evaluate non-ldp docs,
especially ones on Linux in Wikipedia.  The problem with Wikipedia is that
it doesn't allow the original research which some HOWTO's contain.

Most important of all, ldp never came out with any plans to organize linux
documentation in general, and this would involve trying to minimize the
duplication of effort that happens when various linux distributions write
documentation on the the same topic.

Another question is: was not Poet (who advertised his business on his
linuxdoc.com site which also mirrored linuxdoc.org) when he said we should
accept docs in html?  Who needs the other formats?  If one needs text,
it's trivial to convert html to txt.  Accepting docs in html also means
accepting docs in a format that generates html (linuxdoc, wikis, docbook,
etc).

David Lawyer

Le 15/01/2016 06:51, David Lawyer a écrit :
> I think it's about time to update the history of LDP and perhaps be more
> honest about it.  I was sent to the Montreal expo in 2000 and almost
> everyone visiting the LDP booth there complained about our documentation
> being out-of-date.  After that, the problem only got worse except that we
> started reviewing docs more which improved the quality of some docs.
>
> My view is that we made a big mistake by going with DocBook.

One thing is sure, when I tried to revive the LDP the manpower was too 
small to fill the task.

But the same is for my local Linux User Group, that survive somewhat 
with a handful people and 3 or 4 happy visitors (physical) a month.

LDP was necessary at the moment, it's no more now. The way Distributions 
changed from Linux-XXX to XXX (linux) didn't make the things better.

but this is simply life. Linux won the race in embedded devices and 
probably also on servers, but didn't on desktops.

My guess is that LDP should stop working officially, remove any mirror, 
keep the web site open as long as hardware permits (with clear notice 
"for historical purpose only") and may be like you began to do keep only 
living the history page.

as a funny tip, one may know that Ghylhem Aznar was from the same French 
LUG (http://culte.org) as I was (and where I still work), so LDP come 
partly from Toulouse, France and more precisely it's suburb, Ramonville :-))

jdd

On Fri, Jan 15, 2016 at 12:51 PM, David Lawyer ####@####.#### wrote:

> My view is that we made a big mistake by going with DocBook.  Part of this
> was my fault for being partially ignorant of the situation along with our
> leader, Guylhen, also not understanding it and thinking that one could
> simplify DocBook by using only a subset of it.  I was somewhat of a coward
> for not being more strongly opposed to DocBook.

I think it was a good decision those days, but we missed an
opportunity to add more languages when it was technically possible.
And the biggest opportunity was to join github-like service earlier to
replace humans on admin side with robots.

It's very likely, that with support with one of discuss members who
wants to stay anonymous by now, we will be able to fully automatize
submission process and add all documentation formats available via
pandoc, and there is a huge amount of them: markdown,
reStructuredText, textile, HTML, DocBook, LaTeX, MediaWiki markup,
TWiki markup, OPML, Emacs Org-Mode, Txt2Tags, Microsoft Word docx,
LibreOffice ODT, EPUB, or Haddock markup.

It's better late than never, and I do not think LDP should cease to
exist, just modify submission process and start thinking about
developing methods allowing readers to review documents.

> Do we have good archives of the "discuss" list that one can search with a
> search engine?  I kinda doubt it.  I don't think that Google has the
> archive in it's index either since I've done some test searches and come
> up with nothing.  But I've saved most of my outgoing emails so I have some
> info to use.

It's absolutely not a problem, this is full archive from today [~7MB]:

http://lists.tldp.org/discuss-archieve.tbz

S.

>Publishing ones work on ones own website is not always a good idea, unless
>there is someone available to take over the doc if the author is no longer
>able to maintain it.  And this happens a lot.  People get busy with other
>activities and sometimes even die.

Recently, or within the past year or two, I've been dealing with apparently 
arrogant and young people who tend to be anti-social with their particular 
political agenda, while exploiting their distribution's domain names within 
their email addresses.  (ie. Someone's Pythonic culture is better and faster 
than our lower-level or equivalent language.)

Choice I have, either keep fighting and complaining while not being provided 
the required leverage by the younger owners of the companies and organizations, 
or just publish my writings to my own domain.  Why continue aiding a crooked 
culture?

So far, I see Wikipedia on-the-ball with these kinds of anti-social agendas or 
so-called attacks.  Sort of a waste of time being a baby sitter, but this has 
to be done or else people start fighting over nothing.  It's one of those 
things that sound tedious and time consuming, but certain people are trained to 
deal with and the situations are rather extremely easy to solve if people are 
trained to deal with such issues.

>Another was the wiki problem.  People would be more likely to edit docs if
>one could just do it without registering.  But it takes a lot of effort to
>stop spam, including blocking ranges of leased urls that generate spam.
>If ldp couldn't find the people to deal with this, perhaps ldp shouldn't
>have a wiki.  But ldp could have tried to evaluate non-ldp docs,
>especially ones on Linux in Wikipedia.  The problem with Wikipedia is that
>it doesn't allow the original research which some HOWTO's contain.

I tend to agree too, as documentation is sometimes submitted via liaisons, or 
somebody whom has intimate knowledge of a piece of hardware, but needs to 
remain anonymous; not due to criminal prosecution or 3rd party licensing 
conflicts, but for possible future contracts with such companies such as 
Microsoft.

>Another question is: was not Poet (who advertised his business on his
>linuxdoc.com site which also mirrored linuxdoc.org) when he said we should
>accept docs in html?  Who needs the other formats?  If one needs text,
>it's trivial to convert html to txt.  Accepting docs in html also means
>accepting docs in a format that generates html (linuxdoc, wikis, docbook,
>etc).

I've also written a few instructionals in the early days on other websites, and 
completely agree.  If a person cannot simply use text, using Wiki or docbook is 
not going to improve their writing any more.  (All instructionals tend to 
originate from a sketched text-only format.)  Writing text is the initial and 
likely most essential step for writing an instructional.  However (and sadly), 
Google is not engineered to categorize *.txt files into their search engine 
database, tending to require *.html type files.

A large number of writings for the past centuries only had text!  Not too 
further mention, some of the Wiki headings and indenting has become really 
horrid when viewing on a monitor or display.  How about just doing text with 
some sort of version control system? ;-)  Text being extremely easy and first 
hand knowledge, while something like Git provides security and monitoring of 
the files.


-- 
Roger
http://rogerx.freeshell.org/

On Fri, Jan 15, 2016 at 09:50:07AM -0500, Roger wrote:
> David Lawyer wrote 
> >Publishing ones work on ones own website is not always a good idea, unless
> >there is someone available to take over the doc if the author is no longer
> >able to maintain it.  And this happens a lot.  People get busy with other
> >activities and sometimes even die.
> 
> Recently, or within the past year or two, I've been dealing with apparently 
> arrogant and young people who tend to be anti-social with their particular 
> political agenda, while exploiting their distribution's domain names within 
> their email addresses.  (ie. Someone's Pythonic culture is better and faster 
> than our lower-level or equivalent language.)
> 
> Choice I have, either keep fighting and complaining while not being provided 
> the required leverage by the younger owners of the companies and organizations, 
> or just publish my writings to my own domain.  Why continue aiding a crooked 
> culture?
> 
> So far, I see Wikipedia on-the-ball with these kinds of anti-social agendas or 
> so-called attacks.  Sort of a waste of time being a baby sitter, but this has 
> to be done or else people start fighting over nothing.  It's one of those 
> things that sound tedious and time consuming, but certain people are trained to 
> deal with and the situations are rather extremely easy to solve if people are 
> trained to deal with such issues.
> 
> >Another was the wiki problem.  People would be more likely to edit docs if
> >one could just do it without registering.  But it takes a lot of effort to
> >stop spam, including blocking ranges of leased urls that generate spam.
> >If ldp couldn't find the people to deal with this, perhaps ldp shouldn't
> >have a wiki.  But ldp could have tried to evaluate non-ldp docs,
> >especially ones on Linux in Wikipedia.  The problem with Wikipedia is that
> >it doesn't allow the original research which some HOWTO's contain.
> 
> I tend to agree too, as documentation is sometimes submitted via liaisons, or 
> somebody whom has intimate knowledge of a piece of hardware, but needs to 
> remain anonymous; not due to criminal prosecution or 3rd party licensing 
> conflicts, but for possible future contracts with such companies such as 
> Microsoft.
> 
> >Another question is: was not Poet (who advertised his business on his
> >linuxdoc.com site which also mirrored linuxdoc.org) when he said we should
> >accept docs in html?  Who needs the other formats?  If one needs text,
> >it's trivial to convert html to txt.  Accepting docs in html also means
> >accepting docs in a format that generates html (linuxdoc, wikis, docbook,
> >etc).
> 
> I've also written a few instructionals in the early days on other websites, and 
> completely agree.  If a person cannot simply use text, using Wiki or docbook is 
> not going to improve their writing any more.  (All instructionals tend to 
> originate from a sketched text-only format.)  Writing text is the initial and 
> likely most essential step for writing an instructional.  However (and sadly), 
> Google is not engineered to categorize *.txt files into their search engine 
> database, tending to require *.html type files.

But Google does incorporate them into its database and one can search on
words contained in the text document and find it.  I know because I have a
few txt docs on my personal website and Google's "Webmaster Tools" show
that some people are finding them via Google.  I should convert them to
html and see if I get more traffic for them (-:
> 
> A large number of writings for the past centuries only had text!  Not too 
> further mention, some of the Wiki headings and indenting has become really 
> horrid when viewing on a monitor or display.

> How about just doing text with some sort of version control system? ;-)
> Text being extremely easy and first hand knowledge, while something like
> Git provides security and monitoring of the files

I agree but one nice feature about linuxdoc (seldom used anymore) or wikis
is that they automatically create a table of contents.  If there is no
such table in a doc, one can as a substitute type in words to search the
doc but a table of contents is sometimes easier to use to find what you
are looking for (or for something of interest you were not looking for):.
> 
> 
> -- 
> Roger
> http://rogerx.freeshell.org/
> 
> ______________________
> http://lists.tldp.org/
> 
> 
			David Lawyer

On Fri, Jan 15, 2016 at 08:16:20AM +0100, jdd wrote:
> as a funny tip, one may know that Ghylhem Aznar was from the same French LUG
> (http://culte.org) as I was (and where I still work), so LDP come partly
> from Toulouse, France and more precisely it's suburb, Ramonville :-))

But actually Ghylhem didn't participat much in the discuss list and others
posted much more than he did.  He was mainly a leader in "title" only
except for a time when he first took over the leadership due to no one
else being nomitated.  (Poet volunteered but no one wanted him enough to
nominate him.)
			David Lawyer

>But Google does incorporate them into its database and one can search on
>words contained in the text document and find it.  I know because I have a
>few txt docs on my personal website and Google's "Webmaster Tools" show
>that some people are finding them via Google.  I should convert them to
>html and see if I get more traffic for them (-:

mmmm.  Thanks for the verified info.  Glad to hear I'm likely wrong, and will 
likely publish my computer related documents in text only and see what happens.  
Besides, the usual people interested in such data usually are very fluent with 
a text editor.

-- 
Roger
http://rogerx.freeshell.org/

On Fri, Jan 15, 2016 at 02:35:15PM +0700, Serge Victor wrote:
> On Fri, Jan 15, 2016 at 12:51 PM, David Lawyer ####@####.#### wrote:
> 
> > My view is that we made a big mistake by going with DocBook.  Part of this
> > was my fault for being partially ignorant of the situation along with our
> > leader, Guylhen, also not understanding it and thinking that one could
> > simplify DocBook by using only a subset of it.  I was somewhat of a coward
> > for not being more strongly opposed to DocBook.
> 
> I think it was a good decision those days, but we missed an
> opportunity to add more languages when it was technically possible.
> And the biggest opportunity was to join github-like service earlier to
> replace humans on admin side with robots.

But it's my understanding that LDP had scripts which automatically
processed submissions.  And submissin by authors was either via a CVS
account or via "submit" with no account.  What we needed (and still need
is a CMS=Content Management System).  It would establish categories for
how-tos, etc. and a how-to could belong to multiple categories.  I
suggested a number of categories such as doc. being maintained but
maintainer wants to quit.  Our CMS was to be the plone flavor of zope and
David Merrill worked on it but became ill before he implemented it for
LDP.  He said he was incorporating most of the code writing he did for LDP
into Plone but no one else came forward to study it and implement it.
It was to include wiki editing also.
> 
> It's very likely, that with support with one of discuss members who
> wants to stay anonymous by now, we will be able to fully automatize
> submission process and add all documentation formats available via
> pandoc, and there is a huge amount of them: markdown, reStructuredText,
> textile, HTML, DocBook, LaTeX, MediaWiki markup, TWiki markup, OPML,
> Emacs Org-Mode, Txt2Tags, Microsoft Word docx, LibreOffice ODT, EPUB, or
> Haddock markup.

I see neither linuxdoc nor plain text in the above list.

> 
> It's better late than never, and I do not think LDP should cease to
> exist, just modify submission process and start thinking about
> developing methods allowing readers to review documents.
  
 David Lawyer

On 2/2/2016 10:16 PM, David Lawyer wrote:
> On Fri, Jan 15, 2016 at 02:35:15PM +0700, Serge Victor wrote:
>
>> It's very likely, that with support with one of discuss members who
>> wants to stay anonymous by now, we will be able to fully automatize
>> submission process and add all documentation formats available via
>> pandoc, and there is a huge amount of them: markdown, reStructuredText,
>> textile, HTML, DocBook, LaTeX, MediaWiki markup, TWiki markup, OPML,
>> Emacs Org-Mode, Txt2Tags, Microsoft Word docx, LibreOffice ODT, EPUB, or
>> Haddock markup.
> I see neither linuxdoc nor plain text in the above list.

  * Custom formats: custom writers can be written inlua
    <http://www.lua.org/>.


-Mark

Greetings David,

Thank you for your thoughts.  You have been here in LDP for a long 
time, and it is good to have your contributions.

>> > My view is that we made a big mistake by going with DocBook.  Part of this
>> > was my fault for being partially ignorant of the situation along with our
>> > leader, Guylhen, also not understanding it and thinking that one could
>> > simplify DocBook by using only a subset of it.  I was somewhat of a coward
>> > for not being more strongly opposed to DocBook.
>> 
>> I think it was a good decision those days, but we missed an
>> opportunity to add more languages when it was technically possible.
>> And the biggest opportunity was to join github-like service earlier to
>> replace humans on admin side with robots.
>
>But it's my understanding that LDP had scripts which automatically 
>processed submissions. 

Sort of.  This is where I have made my first efforts.  The LDP had 
and still has (some of) the scripts.  Some are visible here:

  https://github.com/tLDP/LDP/tree/master/LDP/builder

See (more) comments on automation below.

>And submission by authors was either via a CVS account or via 
>"submit" with no account.

Correct.  Remains largely correct.  The only real change is that 
authors will now send us pull requests via github.com, or patches 
via ####@####.#### (BUT please wait, we're not quite ready for 
that yet).

<digression>
Mark Komarinski has added a bit to the LDP-Author-Guide about git 
(git.xml) and how we will (eventually) be ready to accept patches 
via email for those who do not wish to use github.com.
</digression>

>What we needed (and still need is a CMS=Content Management System).

Yes.  Agreed.  Some sort of management tool would help tremendously.  

To my way of thinking, this is one of the problems jdd was tackling 
by developing the LDP wiki.

>It would establish categories for how-tos, etc. and a how-to could 
>belong to multiple categories.  I suggested a number of categories 
>such as doc. being maintained but maintainer wants to quit. 

Yes.  Absolutely.

>Our CMS was to be the plone flavor of zope and David Merrill worked 
>on it but became ill before he implemented it for LDP.  He said he 
>was incorporating most of the code writing he did for LDP into 
>Plone but no one else came forward to study it and implement it. It 
>was to include wiki editing also.

Unless somebody wishes to take over Lampadas (using Plone), I think 
we should reassess what our goals are in choosing some sort of 
management system.

>> It's very likely, that with support with one of discuss members 
>> who wants to stay anonymous by now, we will be able to fully 
>> automatize submission process and add all documentation formats 
>> available via pandoc, and there is a huge amount of them: 
>> markdown, reStructuredText, textile, HTML, DocBook, LaTeX, 
>> MediaWiki markup, TWiki markup, OPML, Emacs Org-Mode, Txt2Tags, 
>> Microsoft Word docx, LibreOffice ODT, EPUB, or Haddock markup.

Haddock?!  That sounds fishy!

>I see neither linuxdoc nor plain text in the above list.

Multiple formats
----------------
I don't know much at all about pandoc, but if it allows us to 
accept documents in more formats, then this is a good thing.

As far as accepted submissions, I'd be inclined to say the 
following:

  LDP accepts documentation in almost any format, however, we prefer
  Linuxdoc, DocBook XML 4.2, __new_format_A__ and __new_format_B__.

Accept any format.
Convert to supported format.

At least one reason for creating documents in a structured markup 
language is to enable automated extraction of the structured data.  
That's why there has been a proliferation of document formats (and 
it is unlikely to stop).

I reach back to the manifesto and assert that, yes, we can accept 
high-quality plain text documents.

I would also, volunteer to convert such documents to one of our 
supported (more structured) formats.  The main point is not to 
reject a document because of format.


Automation
----------
While there were tools in our version control system, it was not the 
complete toolchain.  Others were in Greg Ferguson's working 
directory.

As nearly as I can figure (without bothering him directly), the 
process went like this:

  1. check in submissions to ####@####.#### (if emailed)

  2. on document build box, check out document(s) from CVS

  3. run ldp_mk (and ldp_cp? (and ldp_index? (and mk_sorted_ht?)))

  4. copy into place (or did ldp_cp do that?)

So, the publication process appears to have been partially 
automated.

I reconstructed the document handling process by examining both the 
contents of the ./builder/ directory and a snapshot of the working 
directory from the buildbox.  The main script appears to have been 
ldp_mk, which produced the various HTML, PDF, PS and text outputs 
from the Linuxdoc and DocBook formats (there was no support present 
for the WikiText format in this script).

In terms of documentation (on publishing the documentation, heh), 
there is text in guide/docbook/LDP-Admin-Guide, from which I learned 
very little about the mechanical process of publication.  I needed 
to read the Perl to understand.

Final comments on the automation issue(s):

While the tools that Greg Ferguson had collected, built, revised and 
used for a decade worked, I thought it a good time for review.

  * In the years since 2001 (or so) when many of the existing 
    scripts were written, the platform distributors (think, Debian, 
    RedHat/Fedora, SuSE/openSUSE) DocBook have made vast strides in 
    providing solid integration for Linuxdoc and DocBook processing 
    tools.  I wanted to take advantage of these improvements.

  * I desire tools that requires a minimum of local customization:  
    the current tools appear to have lots of hard-coded references 
    to local/homedir filesystem paths for XSL, DSSSL, SGML handling, 
    and even htmldoc and html2text tools.

    Of course, it worked very well for years.  But, rebuilding the 
    toolchain it on a new server could be (or could have been) a 
    headache.

  * I desire a completely automated system (that can run as a
    daemon or from cron or something).  An automated system doesn't 
    go on vacation, which any one of us is likely to do at some 
    point.

  * The existing tools handled not just our submissions, but also 
    LDPWN (LDP Weekly News), so there would have been some 
    untangling anyway.

  * After the untangling, I suspect that it will be easier (from a 
    mechanical perspective) to support new documentation formats.

More soon,

-Martin

-- 
Martin A. Brown
http://linux-ip.net/