« Microsoft's SPOT | Main | The curse of massively multiplayer immersive games »

Tuesday, 11 February 2003

Inflection Point

We've come to depend on applications that can undo everything they do. In Microsoft Word, for example, you can reverse a series of edits by leaning on Control-Z, and you can restore them with Control-Y. But when you save the file, your undo stack evaporates and the last edit wins. If you accidentally delete the file, you might be out of luck because the Recycle Bin only works when applications (primarily Internet Explorer) use the Windows shell API. If you accidentally overwrite the file, you're almost certainly out of luck. [more, Infoworld]

More and more people are re-thinking the way that we interact with computers: the latest (heretical, to some) idea is to further abstract the filesystem away from the user. We've already abstracted it in one sense: while some of us enjoy working in shell environments, the rest of the world is perfectly happy when presented with a GUI shell that depicts filespaces pictorially.

This first level of abstraction has been with us for a terribly long time, and only now have slight improvements begun to take place. When confronted with a folder full of audio files, Windows XP will subtly change the presentation of the folder and offer you more pertinent information, such as track names. This works with images as well: a folder full of JPEG files will switch the display to some sort of photo album and allow you to perform rudimentary operations on the images.

What we're really waiting for is a further level of abstraction. At this point in time, unless you're some sort of die-hard using BeOS, your filesystem is plain-old hierarchical. You have a root directory that can contain files and directories. Those directories themselves can contain files and directories, and so on. You might say that we've stuck with this method of storage because it's easy, and in a way, it is. It's a metaphor that hopefully not many people will have a problem with: the icon for Windows' file manager was a filing cabinet, after all, and it made sense--people used computers in offices to make files, and files in offices were stored in filing cabinets. Folders for this, folders for that, all atomised, all categorised.

Or not.

It takes time, effort and planning to categorise and file, otherwise anybody could be a librarian. Fine, we're not talking about having your own personal taxonomy (although I won't divulge how far I am along that line myself), but from anecdotal experience, the problem with having a "My Documents" folder is that it pretty much contains "My Documents" with no semblance of organisation whatsoever.

It seems that the mere effort of creating a folder and doing the most minimum form of categorisation is too much; and typical users are storing more and more files all the time. This lack of categorisation and organisation is not a failing of the user, it's just a result of bad design. There’s an argument for saying that we’re not particularly good at the model with which we’re being forced to categorise and organise information on computers--in an outline-alike hierarchical manner, and that we really should be able to find a better way.

Enter metadata, stage right.

You already use metadata--data about data--every day: one way of interpreting the email that you receive every day is that the message itself is the data, but the associated metadata is comprised of fields such as who it's from, what address it was from, what address it was sent to, when it was sent, what time zone it was sent from, what it was in reply to and so on. If you sort your email by date, you're using metadata. Our filesystems as they are now also have metadata--data such as when a file was created, when it was last accessed and how big it is. All these kinds of metatadata--the data in your email, the rudimentary data that we have now with files is what might be called weak metadata.

Many still pine for the filesystem of the Be Operating System which maintained a much larger set of metadata for each file that you could navigate using something called a Tracker. So, while you might give that letter to your aunt thanking her for your Christmas present a name like "Christmas letter to Aunt May.doc", the metadata about that file could comprise a whole list of things like:

  • how many pages long it was
  • who it was to
  • when you wrote it
  • that it was a letter
  • that you printed it out

Items such as these, which also included user-definable fields, can be termed as strong metadata--over and above the amount that has historically been used. In principle, there’s no real limit to the amount of metadata that you can store about a file--and later we'll see that we can leave behind the idea of atomised files in their entirety--the only practicable limit being that you should really only store information that's going to be useful. The challenge will be deciding what information will be useful. Note that this challenge is not a great detraction - what's important here is the change in storing the metadata in the first place.

Once we start storing information like this, extracting the metadata about files into a form that we can do useful work with, we come across the concept of Intertwingularity, and search operations like:

  • "find me all the letters I printed out last month"
  • "find me all the letters I've written"
  • "find me everything to do with Aunt May"

become trivially easy, because right now, we can't do anything like that based on the data we're storing about our files.

This is not new: BeOS did this, and many lament that operating system's passing. Microsoft is moving towards a SQL-based filestore [1, 2] that (hopefully) will incorporate slews of metadata about your information. Photographs taken with digital cameras come with embedded EXIF data that allow image editing applications to know such values as exposure and even camera orientation at the time the picture was taken. Metadata is useful and metadata is important. Making metadata accessible and understandable to machines is one way in which we can have more useable and more useful tools.

I won't rehash existing ideas about how wonderful metadata based filesystems are and how they'll change our lives, suffice to round off with a few examples:

  • "soft" folders, whose contents are dynamically updated like smart playlists in iTunes
  • more intuitive icons with higher information density: why is the icon representing a 1 page text document the same as that representing a 200 page text document?
  • likewise, why does a brand new file have the same icon as a two year old one?
  • never having to worry about filing documents in the right place
  • ... and so on

How else can we improve the way that we work? The first time we use the "undo" command in an application after a particularly disastrous bout of feverished editing, we're liable to fall down and kiss the ground, then proclaim our allegiance forever to whomever thought that feature might be a good idea. Undo is wonderful. Undo is our friend. And undo, undoubtedly, is nowhere near as powerful as it can be right now.

Undo is evolving in a number of ways: it's spilled over from document editing into operating systems where we now have concepts such as the Last Known Good State. Installed a driver that you shouldn't have? Fear no more, we can always roll back. In principle, this is a wonderful idea, though perhaps not implemented in a watertight manner in Windows.

There's room for undo to grow, though. With today's desktops shipping with gargantuan hard drives, nearing at least a gigabyte of RAM and with processors hitting 3GHz plus, there's no excuse for trying to extending the undo function as far as we can throw it.

Annoying current limitations of undo that we can fix:

  • not being able to undo past a save point
  • not tracking more than one history of undo

Currently, the action of saving and overwriting a document has an element of finality about it. Consider the following scenario:

  • open your word processor
  • open letter to Aunt May
  • decide to do some editing
  • edit a paragraph by adding a few sentences, undoing a few, adding some more
  • save
  • close your word processor

A while later, you change your mind and want to re-edit your letter to Aunt May. You want to keep one of the sentences that you added, but subsequently deleted before you saved it:

  • open your word processor
  • open letter to Aunt May
  • scratch head

You can't undo past your save point.

There's one way we can solve this problem: by adopting an attitude of scant disregard towards thrift in storage. Consider what would happen if we kept a rolling record of everything that had been done with files in a CVS-alike manner: now we can undo past save points. In fact, we can do some rather nifty things.

In one mode of editing our mammoth every-version-ever Letter to Aunt May, there's a scrolling timeline at the top of the document window. At the extreme right hand side is "now", and at the extreme left hand side is when your letter was first created. We can scrub through this timeline and watch every single change that was made to the document. We can even scan through its history to find changes that were made at a specific time.

The timeline-scrubbing is but one way we can represent the temporal aspect of a file's life. What we've been concerned with so far has been WYSIWYG with no regard to the actual history behind What You Saw: another way of visualising this mass of historical data is as a tree, with large edits becoming branches that may or may not be merged back with the trunk at a later date. A further way is as a composite: documents constructed from blocks that change over time.

Why do we have to manually initiate the save process? We're already creating a rolling record of all the alterations done to our document: can't we just spool this all to disk? It's not as if our disks aren't big enough anymore: we're now down to a dollar per gigabyte of storage. This is not a new suggestion; it's been complained about and continuous spooling has been mooted by others for a while. Let me try and explain how much little sense having to manually save a document makes in this day and age:

When you write something down on a piece of paper, you're not expected to laminate it or perform some other bizarre closing ritual before you file it away in your desk drawer. You would be surprised if you wrote down a note to yourself and when you went back to it the next day, half of it was inexplicably missing after you swear blind you remember writing it.

Similarly, if I close the lid on my laptop right now, with this document open and a myriad of other windows that I have work in, provided I open it up before the battery runs out in a couple days, everything will be just as it was when I closed the lid. But for some reason, should I forget to save the file that I'm working on right now and close the application, everything's gone. Something is not quite right here: there is nothing that should stop saving from being a spooling operation.

Once we introduce the concept of spooling saves, we come up against problems that have to be resolved in the UI. We know that certain types of metadata are desirable, but what we now have to reconcile are the mechanisms by which we can collect that data in the first place.

We have "free" metadata in the sense that the user doesn't have to do anything above what she normally does to supply it. This kind of metadata can take the form of editing times, creation times, indexed contents of the file and so on.

On the other hand, we have "pay for" metadata, which we would traditionally supply at a save point: when we choose to save a document on a system that has a metadata based filesystem but which does not have spooling save capacity, at that point we would expect to be asked for categorising data: metadata such like a short description of the file and a few keywords.

The problem now is that the spooling save capability removes our pre-existing cue in the UI--the manual act of saving--for gathering metadata. We now have to look for new cues that can illustrate some semblance of finality in the document--Matt Webb suggests that these cues can be interpreted as inflection points.

This marks a move away from documents as end-results to documents as works-in-progress--to activities. The move in this direction has already begun; Steven Berlin Johnson has already pointed out the shift to task-based applications like iLife. If we interpret your letter to Aunt May as an activity, we can get some clue as to what kind of inflection points we can reasonably expect: actions such as printing out the letter or emailing it can be interpreted as acts producing finality.

By no means is the above intended as a complete solution. There are many problems that must be resolved by attempting to work with this model of spooling save:

  • is there a way in which the UI can be harnessed to make better use of this new way of working than a simple scrubbing of a timeline?
  • how can we represent the presence of an inflection point in a document's history?
  • given that an inflection point provides a natural event at which "pay-for" metadata will be collected, how can a UI provide a seamless way to collect that data from the user?
  • how now do we handle the concept of a document, given that a particular instance of a document may now be a node in the tree of another document?
  • how also do we deal with branching trees of histories? How can they be merged back into a trunk?
  • how do we search for a document, and when we do search, how do we distinguish between various histories?

This is but a small glance at what we can do and what we can expect. Microsoft's OneNote is an early example of approaching this problem of categorisation and organisation, but there's a way that we can go. Fortunately, we're in the position now of looking for ways to burn off all the idle CPU cycles we've been lumbered with.

6 comments and trackbacks

ext|circ Feb 11, 2003 1:41 PM
I have a new essay up, Inflection Point, which is about--amongst other things--what would happen if we never had to
Read more in Inflection Point »

Is this you Dan, (excuse the 1heluva link on the homepage, long story) My real homepage is http://www.leopardslynx.co.uk

Wow, I bet it is you, oh yeah, it's Emma here, thanks to you I got a B in my A-Level history and we both know that's a miracle!

Sorry to bust in with this totally irrelevant comment but wow! I was just surfing through these blogs and bumped in here by accident!

Blummin' good site by the way

Love ya loads

Emma (and 6 mth old Melody)

Posted by: Emma Jordan on February 11, 2003 07:36 PM

but she's a girl... Feb 11, 2003 9:21 PM
Dan Hon has written a very interesting article about the limitations of the filing cabinet metaphor of computer filesystems, and the about the finality of saving a document.
Read more in Trees, trunks and metadata »

There's no reason why you can't integrate a kind of seamless RCS/CVS element into standard document editing: it's not as if you need some kind of SQL backend for it.

MattW's right, though, about the psychological and cultural necessity of actual honest-to-god rewriting from glances at your nots, rather than re-editing. While the technological solution is attractive for most situations, as a *writing* tool, it makes me a little fearful, as it encourages rejigging rather than rewriting.

(I had an interesting exchange with Donna Tartt back in Oxford about the gruntwork of writing, when she mentioned how she composed her novels in longhand because she feared the way that composing to a screen destroys ones sense of structure. Which is what I felt with my thesis. I also agree with what Ted Hughes once said about what you call "the strange archaic urge to splurge ink all over a page". Pen and ink is elemental, fundamental. It's easier to write a poem with a pen, no matter how many business reports you put together with Word.)

Anyway, for file sorting, it's about bloody time we got an equivalent of the Remembrance Agent -- http://www.remem.org -- on the desktop. Or, the non-desktop: the Minority Report thingy.

Posted by: nick sweeney on February 11, 2003 10:17 PM

plasticbag.org Feb 20, 2003 6:15 PM
There's a great big thing in my head at the moment - all conceptual and wiggling - that I've been
Read more in Thursday Microcontentage... »

ext|circ Feb 20, 2003 8:33 PM
Microsoft's inadvertent release of an Office 2003 beta has enabled a select few (well, MSDN subscribers) to take a look
Read more in OneNote to rule them all »


Email address:


Remember personal info?

Choosing yes will save a cookie on your computer that will fill in your personal details automatically should you wish to leave a comment again.