OSWG logo
OSWG CVS
Information

Project Information
Main Page



open source writers group


This message was sent to the OSWG mailing list by Nik Clayton. The original can be found in the list archives here.

Hi guys,

[ I've cc'd the doc@FreeBSD.org mailing list this on this, but put replies
  back to the OSWG mailing list -- I'd like the FDP to have the opportunity
  to comment on this if there's anything I've missed. 

  For the FDP:  The context of this message is the apparent crisis at the
  LDP, and what to do about it.  One of the things that's been discussed
  is the infrastructure you need in place to support a documentation 
  project, and I thought I'd share some of the technical bits and pieces
  we've picked up over the last 12 months or so. ]

On Wed, Aug 25, 1999 at 07:30:13PM -0400, David Rugge wrote:
> There is a DocBook crash course at this site. I would also recommend
> the FreeBSD Documentation tutorial which has a section on DocBook.
> 
> http://www.freebsd.org/tutorials/docproj-primer/book.html 

Cheers.  It's good to know that people are finding it useful.

> I also wanted to comment on the suggested scheme for automating 
> corrections and updates to documentation. If an email bot is used to 
> update documentation then how do we check to make sure that the changes 
> are useful and worthwhile before they get committed? If an editor needs 
> to do this work then aren't we back to the same problems as with having 
> a documentation maintainer?  Perhaps there could be a weekly rotating 
> team of editors/maintainers so that if a maintainer was unable or 
> unwilling to do his or her part then the work would only pile up for a 
> week for the next maintainer instead of remaining unfinished
> indefinitely. 

You're right, trying to automate updates received by e-mail is a bad idea.

Here's how we do it over at FreeBSD.  I can attest from personal 
experience that this mechanism works.

Put your documentation in to a CVS repository somewhere.  You need some
sort of structure for the directories under this repository, otherwise
things become impossible to manage very quickly.  Over at FreeBSD we've
just finished a fairly large scale move to a new structure to facilitate
the rapid adoption of new documentation with the least amount of pain.

To describe it, briefly;

At the top level, you have two types of directory; language specific, and
non-language specific.  Subdivide your languages by the code for the
language used *and* the encoding mechanism used in that language (this is
because some languages, like Chinese, can be written in more than one 
encoding, and there is not a 1:1 map between them).

These are the vast majority of your directories.  Have one other directory,
share/, and this is where language independent stuff goes.

On the FDP we have English, Spanish, Japanese, Chinese, and Russian 
translations (and French happening very soon), so right now, the 
directory structure looks like:

    % cd ~/FreeBSD-CVS/doc
    % ls
    CVS/              en_US.ISO_8859-1/ fr_FR.ISO_8859-1/ share/
    es_ES.ISO_8859-1/ ja_JP.eucJP/      zh_TW.Big5/       ru_RU.KOI8-R/

Yes, I know those directory names look too long.  What's wrong with just
"en" or "es"?  Trust me, you need to be able to cater for languages that
support multiple encodings (like Chinese), and one day, we'll be supporting
things like Unicode.  So splitting things up by language and encoding *now*
is the way to do it.  You save a *lot* of future pain.

There's also a Makefile in there, but I'm not going to show them.

Take a look inside one of these language directories, like the English
one, and you get

    % cd en_US.ISO_8859-1
    % ls
    CVS/       articles/  books/     share/

So the documentation is divided in to two categories, books, and articles.
There's a third directory, "share", which is for additional support files
that are specific to this language (for example, the boilerplate license
text might live in share/sgml/license.sgml, and be referenced in all the
other documentation as an SGML entity.

Note the categories -- books, and articles.  There are no attempts to divide
the documentation based on the content, but only the purely mechanical 
decision of "Is it a DocBook <book> or an <article>?"  Articles are 
appropriate for smaller HOWTOs and similar documents, books are appropriate
for large bodies of documentation, such as the primer quoted earlier.

IMPORTANT:  The structure of the documentation in the CVS repository *DOES 
NOT* affect the structure of the documentation when it has been installed
on to a user's disk.  You can use various Makefile tricks to ensure that
the same piece of documentation is installed under /usr/share/doc/printing,
/usr/share/doc/win95, /usr/share/doc/articles, and /usr/share/doc/samba --
consider an article explaining how to use Samba to print to Windows 95; a
user might expect to find that article under any of those directories on
their installed system.  You could even have a virtual category of 
"howto", if you wanted.  The point is, you don't want to put these 
categories in the CVS directory structure -- instead, encode it as options
in a Makefile/

So, the purpose of this division is simply to make it very, very, easy to
decide where the documentation lives under the CVS tree.

You could say "Why not put it all at this directory, and not bother with 
the books/ and articles/ distinction?".  That's because there's one 
directory that I haven't shown you here, "man/".  Yes, you can put manual
pages under this infrastructure as well,  so that a completely fleshed
out directory would actually look like

    CVS/       articles/  books/     man/      share/

Then take a trip down in to books/, and you see

    % cd books
    % ls
    CVS/          fdp-primer/   faq/          handbook/

As you can see, this is where the FDP Primer, the FAQ, and the Handbook 
live.  Again, I haven't shown the set of Makefile's that go along with
these.

Finally, in these subdirectories, you have the contents of the books 
themselves.

That's how we structure the repository.  It's clean, simple, logical, and,
as I say, adding new documentation (or new translations) is trivial, because
you have to expend almost zero effort in deciding where in the tree to put
it.  The books/articles/man distinction is simple, and the hardest part is
coming up with the name of the 'leaf' directory (fdp-primer, handbook, etc).

It also scales to new languages and new encodings very easily.

Right, so you have this in a CVS repository somewhere.  You need to make it
easy for anyone to check stuff out of this repository -- that's pretty
trivial, cvs can do it itself, or the cognoscenti use CVSup to make things
go nice and fast.

Now that anyone and everyone can download the source to your documentation,
you need to make it easy for them to install the tools that they can use
to edit the docs, convert them to other formats, and so on.  On FreeBSD
we have the "ports" system, which lets us easily bundle up multiple apps
in a FreeBSD specific manner, and get them on to the end user's system.
Someone that wants to get all the tools they need to produce HTML or RTF
or text versions of the docs just needs to do

    # cd /usr/ports/textproc/docproj
    # make JADETEX=no install

on their FreeBSD system, and come back when it's finished installing.
Alternatively, they can do "... JADETEX=yes ...", and get the ability to
produce PS and PDF output too.

Actually calling these programs to convert the documentation is covered 
by the Makefiles that I didn't bother showing you earlier.  All the
author has to do is "make FORMATS=html" or "make FORMATS=ps" to see what
their documentation looks like.

Of course, most people aren't authors (or potential submitters), they 
just want to get the documentation in a usable format.  In order to do
this, they can download the FreeBSD Documentation Set packages from the
FreeBSD FTP server, with a command like

    # pkg_add \ 
	 ftp://ftp.freebsd.org/pub/FreeBSD/docs/faq-en_US.ISO_8859-1.pdf.tgz

And it will do the right thing (well, almost -- I'm putting the finishing
touches to the infrastructure to support this now, so that's ready
and usable by next week).  These packages are (or will be) built daily.
And because they fit in with the FreeBSD packaging system, the user
can remove outdated ones, and install new ones, with pkg_delete.

On a Linux system I imagine you'd use RPMs or DEBs, or whatever.

So, this tackles the storage of the document source, getting it to authors,
putting tools in the hands of the authors to maintain their documentation,
and easily distributing the formatted documentation to end users.

How about actually making changes to the documentation?  There are two ways
this can happen.

The first is that you give the author an account on the host that holds
the CVS archive, and you let them commit directly to the archive.  This
requires the author to know how to use CVS, and requires you to trust
the authors that they won't inadvertently trash the archive.  For some 
authors this is a problem, for others it isn't.

For those that can do this, this is the simplest method.

The slightly tougher method is through GNATS, the GNU action tracking 
system.  Authors can use send-pr to submit updates to their documentation
(as diffs, ideally) or they can post pointers to the latest SGML archive
that they've put together.  The committers can then process these diffs,
and apply them to the documentation on the authors behalf.

So, that's how authors can submit the changed versions of their documents
back to the FDP, but what about end users, who might have spotted typos?

Well, the end user can always send their comments to the author if they
want, and the author can handle it that way.  Or, the end users can use
exactly the same mechanism as the authors, and use send-pr to submit 
fixes, typo corrections, and so on.  

Simple!

There's one more thing -- with the implementation above, particularly 
the send-pr implementation, you will find that one or two people's
names will frequently crop up in your list of people submitting PRs.  
You'll notice that their PRs are good quality, have accurate diffs, and
always do the right thing.

Press gang these people into joining you as soon as possible, because the
biggest problem with any project like this is lack of manpower.  Give
them the rights to commit directly to the repository to fix things, and
let them handle the incoming PRs.  The FreeBSD Documentation Project is
very fortunate to have volunteers of very high calibre working on it; some
of them have been working on the FDP for years, others only joined in 
the past few months, but I am constantly impressed by the effort they
put in.

[ Here endeth the lesson :-) ]

N
-- 
 [intentional self-reference] can be easily accommodated using a blessed,
 non-self-referential dummy head-node whose own object destructor severs
 the links.
    -- Tom Christiansen in <375143b5@cs.colorado.edu>


Linux is a trademark of Linus Torvalds. All other trademarks are owned by their respective companies.
Everything else, © 1999 Deb Richardson