Off-line Wiki - using TWiki with Subversion

As part of my personal productivity setup I use a wiki, and for a while now I've wanted to be able to use it off-line as well - on a laptop without Internet connection.

I finally took the plunge, and converted my previous on-line-only TWiki into an on-line/off-line setup based on FosWiki. It was remarkably easy. At least, after I finally decided not to use any of the fancy alternatives.

There is a surprisingly large number of experimental wikis in this area, but most do not look very usable or stable. For example I looked at Ikiwiki and Gitit, two Git-based wikis. Both looked interesting but it turned out that neither supports Windows. One of my requirements is to run it off-line on a Windows laptop. I also looked at various extensions for TWiki and FosWiki that are supposed to support off-line work, such as MercurialContrib, Subversive and SubversionStoreContrib. In the end I decided to focus on Subversion (instead of Git and Mercurial) because subversion is simpler. When I tried out SubversionStoreContrib for FosWiki it turned out to be incomplete and didn't work. 

In the end I selected the following straightforward setup:

• First I migrated my on-line TWiki to FosWiki, because FosWiki is easy to install on Windows and more up to date than my aging TWiki setup, while still 100% compatible with the TWiki page format. This will allow me to create off-line copies on various Windows environments easily.
• I imported the pages from the Main web into Subversion, but only the ones I'd written myself. All system pages (such as WebPreferences.txt) were added to the ignore list, as well as all files ending in ,v and .lease. I did the same for the attachments in pub/Main. 
• Next I created two simple shell scripts to update and commit the local svn workspace from/to the repository. These scripts add new files to the repository where needed. They can be called from links on the main WebHome page.
• For the off-line setup I installed FosWiki to my laptop and used TortoiseSVN to check out the page and attachment parts of the repository into the right FosWiki directories. 
• The main trick here is to leave the normal TWiki/FosWiki RCS-based store unchanged, but simply add *,v to the SVN ignore list.
• The result is that most use cases are covered both on-line and off-line; editing, updating, committing and adding new files. I haven't covered deleting and renaming files yet in the on-line scripts because those are pretty rarely used, and can be done easily off-line using TortoiseSVN.
• Another result that is not very elegant is that because the RCS ,v files are not in SVN they are basically worthless and the history view in the wiki itself will be incorrect. I simply chose to ignore this, and when I want to see the page history (pretty rare again) I can do so using the SVN tools.

All in all I'm very satisfied with this setup, it does what I wanted it to, and is suitable for use by one or more users with technical / Subversion experience. It seems less suitable for very large user groups and non-technical users.

If you're interested let me know in the comments and I can supply more details (scripts, ignore lists etc.)

TWiki vs MediaWiki part 2 - the battle

Ah TWiki. It is great. (See part 1 of this post for some background). Some of the things that I really like about TWiki are very subjective, but I'm going to list them anyway:

• Its default interface is nice-looking and not too cluttered (big score against TikiWiki).
• Its file-based storage is easy to understand, and gives me a confident feeling - an admin user with a text editor can access the page data directly to fix things if needed, run scripts, etc. I haven't needed to do that yet, but it's obviously a big plus compared to XWiki and most other wikis. (In case you are wondering - the only major downside as far as I can see is slow search when you have tens of thousands of topics. That bridge is so easy to cross that I don't consider it a downside at all).
• It's very flexible, with a powerful template-based user interface and a solid collection of plugins.

Now I've started some consulting for a new web site that obviously needed a CMS. It's a bit like a Wikipedia, so I started looking at MediaWiki as an obvious implementation candidate. (It also turned out to be the only serious new candidate. There are various wiki feature comparison matrices on the net, and if you skip commercial candidates like Sharepoint and Confluence, and skip Google Sites because it's limited in many ways then it seems MediaWiki, TWiki and TikiWiki are more or less the main contenders). MediaWiki has some nice features, in particular its internationalization support. For many purposes it may be the best wiki around. It certainly is a serious contender. (I suspect that TWiki is more fun to use and look at, but didn't try to prove it).

Two things killed it for our purpose. It doesn't have sub-wiki's, which surprised me. It doesn't have page-level authorization! I know that authorization in general is a bit against the wiki spirit. The TWiki documentation doesn't hesitate to remind the administrator of that at every possible opportunity, but at least TWiki has the feature. This gives the admin the possibility to choose to use it or not. But the MediaWiki guys thought this wasn't a feature we should use, so they didn't build it in! To me this seems like two species of dinosaurs involved in a Darwinian battle for survival, with one of the dinosaurs saying 'oh, I'll pull out my own claws, I really shouldn't use them because they're not nice'. Should be an easy victory for the other guy.

So far TWiki hasn't let us down yet in this new setting. I takes a bit of work to turn it into a public CMS (instead of the free-for-all edit fest that a wiki is by default), but the result is quite satisfying from a technical standpoint. No ugly hacks needed. Yay.

TWiki vs MediaWiki part 1 - wiki prehistory

Over the past few years I've had the opportunity to test out various wikis, for teams within my previous company, Devinition. I've learned that wikis are great, but that even some of the well-known wiki engines are pretty awful. I'll use a few blog posts to write down my likes and dislikes, so you don't have to go down the road that we did - trying out everything under the sun and ending up with a whole bunch of incompatible page graveyards.

We started out with TikiWiki, back in 2005. TikiWiki certainly has tons of features built-in, but I disliked it from the start because it looked like a typical Open Source 0.1 product. (I assume - and hope for its users - that recent versions have better looks).

Then we implemented an XWiki for the whole company. Primarily because it seemed the Java-based wiki with the best features at the time, and I optimistically thought that with all the Java skills in our company we could do some great integration magic with it. Instead it quickly instilled fear with its horrible user permissions interface. We've had to resort to direct access to the underlying SQL database only once, when some colleagues messed up the permissions beyond normal repair with only a few easy keystrokes. Interesting though it was - trying to get the correct settings back again while puzzling through a meta-meta model - it put XWiki on my avoid-in-the-future-at-all-cost list. However, at that point our teams had already put quite a number of pages into it, so it was really too late to migrate away.

As a kind of in-between snack I used Google Docs with some smaller teams. It works great, but it's not really a wiki replacement of course. Instead it has an excellent spreadsheet feature, and it lets you easily share a few documents with random people. My golden tip of the day is for Google: simply add a 'wiki page' document type to Docs with concurrent editing and support for adsense display and it will run circles around Google Sites and 99% of the existing web hosting.

At that point I also started using a personal wiki. It coincided with reading David Allens 'Getting Things Done'. In my experience they are a great match. A personal wiki is easily flexible enough to act as a store for all your GTD administration (project lists, actions, reference data, etc.) Looking at it from the other side - a wiki can quickly become a mess of random pages and the GTD method provided for useful structure. I'll leave the details of my GTD approach for a next post, but just finish by saying that I used a TWiki, perhaps because it was the next well-known Open Source wiki that I hadn't tried yet.

In part 2 I describe why TWiki is better than all the other wikis, and how it beat MediaWiki in a fierce dinosaur-style battle.