So, I've finally done it. I've turned Ishbadiddle into a semantic web, of sorts. 3.75 years of blogging, 2,341 posts, have all been imported, categorized, and coded. (With help from our posters of course!) There are now 1,385 keywords in our subject index, covering everything from Abu Ghraib to Zombies.

Why bother? Wouldn't it just be easier to leave all the old posts alone? Well, of course it would be easier. But I wanted to fix what I see as a fundamental problem with blogs.

At heart, a blog is just a database. (I am indebted to Sun for our many conversations on this topic that helped push my thinking on blog organization.) URLs, post text, authors, comments -- it's all just data, and in theory we can slice it any way we want to. But most blogs only divide up the information by time. Which is useful, perhaps, for a personal diary -- what was I thinking about last June? -- but for a reader, it's probably the least interesting way of reading. When was the last time you casually read a blog's archives? Generally, if it's not on the front page, it's gone.

Worse yet, everything is in reverse chronological order, leading to what Eric Meyer calls the Memento effect: "Reading a weblog is like watching Memento, which I agree was a cool movie, except all weblogs are like that so it's as if every single movie released in the past seven or eight years was structured exactly like Memento."

This is one reason why we moved Ish over to Movable Type. With the old Blogger system, we were restricted to the reverse-chronlogy diary mode by default. MT lets us slice up Ishbadiddle by Author, by Category, and yes, by Month.

The category system is an essential organizational tool -- if you're just interested in current events, or to catch up on your friends, or read our peculiar cultural views, or our thoughts on technology, or just want to read the wacky stuff -- you can narrow down Ishbadiddle into these thematic chunks. Useful? Sure. But at best, it's only creating "sub-blogs" (sublogs?). These still don't get at the fundamental problem with blogs.

The Fundamental Problem with Blogs

The problem is, that the information in blogs isn't organized in a way that gives context to what's been written. If we're writing about Iraq, or turbans, or Buffy the Vampire Slayer, it helps to know where we've come from. A good blog is a like a cocktail party, minus the drinks and the funny paper umbrellas. But blogs only give you the last statement of the conversation. For those of you who use Gmail, think of the difference between receiving a regular email (where the thread is either truncated or reverse-ordered), and seeing your email presented as a conversation.

One solution would be to increase the number of categories. But for a general purpose blog like Ishbadiddle, that number would get unwieldy. (Did I mention that there are 1,385 keywords?)

Why not just use the Search Function?

The search function assumes that you know that there's something to look for. Plus it's imprecise. A concept may not appear in a post, but be directly relevant to it. Or a word may appear in a post but have nothing to do with it. For instance, you'll find a post on Dog Day Afternoon and Harken Energy in a search for "Dog", but not in the subject listing for "Dogs".

Enter the Semantic Web

My thinking about the Semantic Web was influenced by Paul Ford's piece on the subject, which imagines the power of Google harnessing the Semantic Web to make even more money. There's a good article on the Semantic Web on wikipedia. Basically, it's adding metadata (data about the data) to web pages. In our case, it's simply adding "subject" data to each blog post, and then harnessing that to create an index of posts that relate to that subject. Think of it this way: the Category system is like the Table of Contents of a book, listing chapter headings. The Keyword system is like the Index of a book, one that is constantly updated.

The advantage of such a classification system is that it is open-ended and bottom-up. With 19 different posters here, it would be impossible to impose a complex hierarchical classification system. The data's too unruly. There would be "too many notes."

But for the bottom-up system to work, it needs a few simple rules:

• Redundancy. Not every Matrix-related post will also have Science Fiction as a keyword. But if at least one does, then the two subjects are related. Look over on the sidebar, and you'll see the list of related subjects, which is itself just plain fun to play with. With an open-ended scheme, we will end up with subjects that are synonyms, or subjects (Animals) that contain others (Cats.) Redundant keywording ensures that readers are only two jumps away from related posts on any given topic.

• Specificity.Ambiguous keywords only muddy the waters. To be as specific as possible, our policy is:
  
  • Use full names of contemporary figures, so we don't confuse Dick Cheney and Lynne Cheney, for instance. And George W. Bush gets his middle initial to differentiate him from his father. Madonna is of course excepted.
    
  • Historical figures can use one name (Orwell, Kafka, etc.) as long as it's generally accepted to only use one name.
    
  • Capitalization matters, due to some technical issues. Please Use Sentence Case to Post Your Keywords.
    
  • Check the list to see if a word (or a synonym) is already in use.
    

• Chronology. The Subject pages are presented in chronological order, so that the reader can follow the complete conversation.
  

Follow these simple guidelines, and semantic goodness will spread throughout the blog.

Where is this going?

First, the system is entirely too slow. It's using Movable Type's native search function, which is slow to begin with. Ideally we'd create an index which would be much faster, but I lack the programming chops to do this. Second, using the search function gives us some funny results on short keywords -- The Art index also includes posts on Arthur C. Clarke, Art Spiegelman, and Artifical Intelligence. Gotta fix that.

Third, I would really like to have a graphical interface to show the connections between subjects and posts. I've looked into TouchGraph, which has a really neat interface, and can take an XML input. But again with the lack of progamming chops. Fourth, a count of how many posts are assigned to each keyword. Fifth, it would be really cool to geographically map place-related keywords...

But that's all for later. For now, enjoy the blog. Let me know how we can improve the coding. And if you're interested in doing something like this on your own blog, let me know, I'd be happy to help.

Update: I was talking about this with Debbie, who knows a thing or two about archives. She suggested that the full list of keywords is too long to be browsable, but that a list of keywords for each category might be useful for the reader. So, now from the Category list you have the option to either see all the posts, or just browse the subjects covered for that Category. I must say, a brilliant suggestion, and not just because she's my wife.

2nd Update: Peter Merholz argues for "free tagging" as opposed to "inflexible top-down approaches." And LukeW proposes using Tufte's sparklines as a way to visualize the narrative sequence of related blog entries.

3d Update: I added a list of recent subjects to the sidebar. And I put the code recipe up on the Movable Type support forum if you're interested in doing something similar.

One More Thing! The coding also puts the keywords as subject meta-tags on each entry's page, according to the Dublin Core standard.

And again: I've written up a better tutorial on Learning Movable Type.

Name:

Email Address (required but not displayed or shared):

URL (optional):

Remember personal info?

Comments (you may use HTML tags for style):

Captcha:

Type the characters you see in the picture above.