Tag Archives: docbook

Documentation formats and speaker interviews

Posted on by
Reply

In lieu of actual content on this blog, allow me to introduce you to some recent posts I’ve done for Opensource.com:

Deploying Fedora 18 documentation: learning git the hard way

Posted on by
Reply

If you haven’t heard, the Fedora team released Fedora 18 today. It’s the culmination of many months of effort, and some very frustrating schedule delays. I’m sure everyone was relieve to push it out the door, even as some contributors worked to make sure the mirrors were stable and update translations. I remembered that I had forgotten to push the Fedora 18 versions of the Live Images Guide and the Burning ISOs Guide, so I quickly did that. Then I noticed that several of the documents that were on the site earlier weren’t anymore. Crap.

Here’s how the Fedora Documentation site works: contributors write guides in DocBook XML, build them with a tool called publican, and then check the built documents into a git repository. Once an hour, the web server clones the git repo to update the content on the site. Looking through the commits, it seemed like a few hours prior, someone had published a document without updating their local copy of the web repo first, which blew away previously-published Fedora 18 docs.

The fix seemed simple enough: I’d just revert to a few commits prior and then we could re-publish the most recent updates. So I git a `git reset –hard` and then tried to push. It was suggested that a –force might help, so I did. That’s when I learned that this basically sends the local git repo to the remote as if the remote were empty (someone who understands git better would undoubtedly correct this explanation), which makes sense. For many repos, this probably isn’t too big a deal. For the Docs web repo, which contains many images, PDFs, epubs, etc. and is roughly 8 GB on disk, this can be a slow process. On a residential cable internet connection which throttles uploads to about 250 KiB/s after the first minute, it’s a very slow process.

I sent a note to the docs mailing list letting people know I was cleaning up the repo and that they shouldn’t push any docs to the web. After an hour or so, the push finally finished. It was…a failure? Someone hadn’t seen my email and pushed a new guide shortly after I had started the push-of-doom. Fortunately I discovered the git revert command in the meantime. revert, instead of pretending like the past never happened, makes diffs to back out the commit(s). After reverting four commits and pushing, we were back to where we were when life was happy. It was simple to re-publish the docs after that, and a reminder was sent to the group to ensure the repo is up-to-date before pushing.

The final result is that some documents were unavailable for a few hours. The good news is that I learned a little bit more about git today. The better news is that this should serve as additional motivation to move to Publican 3, which will allow us to publish guides via RPMs instead of an unwieldy git repo.

To automate or not to automate?

Posted on by
4

According to HackTux.com, the eighth sysadmin commandment is

VIII. Thou shalt automate

Do more, faster by automating tasks. You are the puppet master, do not waste your time with menial work. Work smarter, not harder.

It makes sense to automate tasks. Automation reduces the number of opportunities for human error and it makes processes repeatable. These are good things, but like any ideological absolute there are exceptions. One issue I’m currently wrestling with is whether or not the builds of internal documentation at work should be automated. There are compelling arguments for both choices, and it’s not clear to me which one wins out.

Since I’m the one who is leading the documentation charge, I got to pick the technology. My work in the Fedora Documentation group has made me familiar with DocBook XML and Publican, so that’s what I chose. The combination of the two allows for a single source (that can be edited in any editor and neatly managed with your version control system of choice) that can be built into HTML, PDF, or epub. While I wouldn’t call publican “fragile”, it can be a bit verbose in its output. Sometimes the output can be helpful, and sometimes it can be easily ignored, and I’ve not yet come up with an easy way to programmatically distinguish the two. More importantly, the output can give a clear indication that the syntax is invalid (although it’s not always easy to figure out what the precise problem is).

The arguments in favor of automated builds are championed by one of the group’s managers. His concern is that even a well-documented build process may be a deterrent to some group members. Automating the builds removes the opportunity to make mistakes and makes the process simpler. On the other hand, it presents additional moving parts which may break in new and interesting ways.

There are two ways to handle the automated builds. First, every time a document is updated in the version control tree, a post-commit hook builds the new version. This has the ability to provide nearly-instant feedback on whether or not the document is semantically valid and has the correct layout and content. On the other hand, it may now require multiple commits to finally get the document correct. The writer could, of course, run publican locally before committing the changes, but if they’re going to do that, why have an automated process repeat the work? This method also contradicts the “commit early, commit often” philosophy that I embrace.

Instead of triggering builds based on commits, the build can be done on a schedule. This prevents multiple concurrent build attempts due to rapid commits, at the expense of losing quick feedback. In addition, it can muddy the “who broke this doc?” waters. If there are multiple commits between builds, it is potentially more difficult to hunt down the offending error (or even worse, errors). It also absolves writers of responsibility — since the entire group must be notified that the build failed, everyone will assume someone else will fix it. As a result, the document doesn’t get fixed.

I’m sympathetic to the pro-argument builds. On the other hand, the Fedora Docs are manually built and published (granted there are other considerations like release cycles and translation). Having to run and correct builds by hand helps train writers to write valid syntax the first time through and it allows documents to be updated when they’re ready, while still giving writers the ability to commit works in progress. Useful documents are almost never write-once, read-many. Even the U.S. Constitution has had 27 patches accepted. Still, for the kinds of documentation intended (e.g. design documents), once a document is initially written, it shouldn’t change often. This suggests that designing an automated build system is over-engineering, especially considering the current staff shortage.

It would seem that my current inclination is to keep with a manual build process, even if it is heresy. Fortunately, this may be a solved problem in the not-so-distant future. Red Hat’s Joshua Wulf (and others) have been developing a live DocBook editor that can be done entirely in the web browser. The idea is to lower the barrier to entry for Fedora contributors. This is still in the proof-of-concept stage, but the initial results are promising. In the meantime, I’ll work on getting the content seeded. If someone else in the group wants to automate it, I won’t stand in the way.