discuss: Thread: thoughts (and more) on step 1, automation

Hello all,

Questions first, then some notes based on observations of current 
state.

Then, finally, my proposal of a change to our toolchain.

>  1. automation: Be able to (re-)process and (re-)publish all of
>     our existing documentation in an automated fashion.

Questions that I'm assuming get resoundingly answered "No, nobody 
cares!":

  * Does anybody object to dropping our support for Plucker,
    a quite rare format targetting PalmOS.

  * Does anybody object to our dropping support for PS files?
    We would, of course, continue to supply PDF files.

    [If anybody actually cares or feels strongly that we should keep 
    supplying .PS files, it is not hard to add that....]


Automation:
----------- 
I have begun a review (and rewrite) of the scripts that consume the 
source documents in the LDP collection source tree (now hosted on 
github.com [0]) and generate the various print, HTML and text 
formats.

Current status (of technical tools):

  * publishing tools use [open]jade wrapped underneath an
    LDP-specific set of perl tools, see:
    https://github.com/tLDP/LDP/tree/master/LDP/builder

  * publishing tools appear to operate on single documents
    with some of the management of files being manual (or I
    have not found all of the tools)

  * The ldp_mk script appears to have been the main tool.  It
    includes code for handling Linux Documentation Project Weekly
    News (LDPWN), unfortunately deceased in 2006-04-05.

I have at my disposal an OpenSUSE-13.2 system (my own) and an 
Ubuntu-14.04.3 system (thank you, Serge).  I am using the 
distributor supplied toolchains in each case, to minimize the need 
for customization.  Notably, the Ubuntu upstream includes a package 
with our stylesheets [1].

Here are my suggested changes to our processing of source documents 
into HTML (chunked and single-page), PDF and text.

  1. Process all DocBook XML documents with an xsltproc and
     xsl toolchain.

  2. We will strongly prefer distribution-supplied packages
     and tools.  This eases the maintenance burden.

  3. The document processing system should always be able to
     rebuild all outputs from the (canonical) source.
     In concert with suggestion 2., regeneration of all outputs,
     is possible by anybody at any time.

Short listing of toolchains:

  - linuxdoc SGML: sgml2html, htmldoc, html2text
  - docbook SGML: openjade (jw), collateindex.pl, html2text
  - docbook XML: xsltproc, html2text, fop (dblatex fallback)

Here, I would introduce a change to TLDP's document processing 
toolchain.  In the past, all of our documents (Linuxdoc, DocBook 
SGML, DocBook XML) were published using the jade/openjade toolchain.  
DocBook XML My automation scripts can publish DocBook XML using the 
xsltproc and xsl stylesheets toolchain.

I have a tested and working toolchain on OpenSUSE-13.2 and 
Ubuntu-14.04.3, thanks to the many years of integration work that 
has happened by a variety of players in the open source arena.  In 
short, the existing (nearly stock) platforms can handle processing 
our DocBook 3.0, DocBook 3.1, DocBook 4.1.2, DocBook 4.2 and 
Linuxdoc files without any monkeying with DSSSL and XSL files [1].

I will keep on working away on other steps in this process,

-Martin

P.S. In case you were wondering, I have been working on this project 
  a bit before I announced anything today.  I wanted to have 
  something to say, before I started blabbering about it.

 [0] https://github.com/tLDP/LDP/tree/master/LDP/builder

     https://github.com/tLDP/      # -- our account
     https://github.com/tLDP/LDP/  # -- main source document repo

 [1] The package is ldp-docboox-xsl contributed by Frank
     Lichtenheld in 2004.  I had to make two changes to the
     XSL stylesheets to get them to work correctly with modern
     FOP, but it is great to see this stuff packaged.

-- 
Martin A. Brown
http://linux-ip.net/

On Tue, 26 Jan 2016 16:54:25 Martin A. Brown wrote:
> Hello all,
> 
> Questions first, then some notes based on observations of current 
> state.
> 
> Then, finally, my proposal of a change to our toolchain.
> 
> >  1. automation: Be able to (re-)process and (re-)publish all of
> >     our existing documentation in an automated fashion.  
> 
> Questions that I'm assuming get resoundingly answered "No, nobody 
> cares!":
> 
>   * Does anybody object to dropping our support for Plucker,
>     a quite rare format targetting PalmOS.
> 
>   * Does anybody object to our dropping support for PS files?
>     We would, of course, continue to supply PDF files.
> 
>     [If anybody actually cares or feels strongly that we should keep 
>     supplying .PS files, it is not hard to add that....]
<snip>

I think ps is still very important for at least three reasons:
1. PS is ASCII (or it can be so), which compresses well.
2. PS is (was), the standard for printing and I, at least, have a PS
capable printer.
3. PS is, like pdf, all one file and some viewers keep tabs on where you
closed it (pdf is the only other format that supports images and has
viewers that do so (edit: djvu viewers might also)).
4. Often times I find an error (typo), in the file and I can correct it
in the PS doc (before reporting it and praying that the maintainer
is listening/alive/willing to fix it), pdf does not allow this AFAIK.

Sincerely, David

Hello there,

>>   * Does anybody object to our dropping support for PS files?
>>     We would, of course, continue to supply PDF files.
>> 
>>     [If anybody actually cares or feels strongly that we should keep 
>>     supplying .PS files, it is not hard to add that....]
>
>I think ps is still very important for at least three reasons:
>1. PS is ASCII (or it can be so), which compresses well.
>2. PS is (was), the standard for printing and I, at least, have a PS
>capable printer.
>3. PS is, like pdf, all one file and some viewers keep tabs on where you
>closed it (pdf is the only other format that supports images and has
>viewers that do so (edit: djvu viewers might also)).
>4. Often times I find an error (typo), in the file and I can correct it
>in the PS doc (before reporting it and praying that the maintainer
>is listening/alive/willing to fix it), pdf does not allow this AFAIK.

OK, great.  One vote for retaining .ps.

While examining our earlier publication tools, I observed that 
there's a discrepancy between their mention of PostScript 
generation in several places and the output tree, which has 
practically no PostScript.

I'll wait for another vote or two, but it's not hard to support an 
alternate output format.

Thanks for your input!

-Martin

-- 
Martin A. Brown
http://linux-ip.net/