${me:-whatever}

As the kids say “epic fail.” I’ll start with that so that hopefully my highly radioactive core of curmudgeon doesn’t shine through quite as brightly.

Now I’ll tell you a bunch of stuff you already know. Email works like this:

1. A user fires up an email client and composes an email and clicks “Send.”
2. The email client connects to its configured SMTP server and hands off the message.
3. The email server looks up the MX (that’s Mail Transfer) DNS record(s) for the recipient(s), connects to the mail server(s) and hands off the message.
4. The recipient(s) fire up their email clients and connect to their configured POP/IMAP/etc servers and retrieve the message.

It makes sense. It’s simple. It scales (just ask spammers!) I can only speculate why people don’t like it. Maybe email clients are terrible. Who am I kidding email clients are terrible. Even the best of them are awful for any number of reasons. But I’ll wager it’s more than that.

More stuff you already know: People blog, corporations blog, everybody blogs. Even if it’s not a blog, one still needs a way to notify one’s constituency that “I put something new on my website!!” No one wants to go visit a site over and over only to find nothing has changed. So feeds and feed readers were born. And to me, at least, this was a somewhat confusing turn of events.

Email, you see, is capable of sending HTML email and many, dare I say “most” email clients can render HTML email. If it’s not painfully obvious where I’m going here: it would seem quite logical to send an email to interested readers every time ones site is updated. Heck, it could even contain the full HTML of the update if you wanted. That way, blogs could work just like mailing lists of yore. Readers could comment on your blog entries via email. Things don’t work that way, though. They work like this:

1. I write my snarky whiny blog entry and click Save in my blog software.
2. The blog software converts my blog entry to at least one, but probably two different formats (RSS/ATOM.)
3. Optionally, the blog software might have a second set of feeds for comments. Doing this same double-conversion on them as well.
4. Should someone be interested, they use a feed reader to subscribe to one of these feeds.
5. Their feed reader makes repeated requests to my blog software asking if anything is new.
6. “Asking if anything is new” gives this solution too much credit. What it really does is ask about caching information (And that’s if the blog software has implemented caching.) and if the cache time-to-live has expired, it downloads the entire feed from the site, not just what’s new.
7. Then it goes through the whole feed comparing it to what its already seen (a process that’s more difficult than you might think) and if something is, in fact, new it alerts the user that something is new.

Wow, that’s a lot.

The worst of these feed readers are really nothing more than glorified bookmarks. The names of the sites and maybe a favicon are presented in a list. The feeds that have unread entries have a bold title and a number of unread items next to the title. These are the worst because they go through all of the above nonsense for next to no value.

For readers like this, feeds could simply be an integer. If the last number you got from my site was 110 and now it’s 112, you know to show your user articles 111 and 112. In fact, this alone would be a huge improvement on the current system because feeds all have their own bandwidth. Since my feed doesn’t know when the last time you asked for it was, it can’t possibly know how many things have changed since you last asked. If I’ve updated 10 things, but my feed only contains 5, you’ve missed 5 updates.

Only slightly less awful than the bookmark style feed reader is modeled after, you guessed it, an email client. This feed reader combines feeds into a meta-feed that’s presented in some variety of chronological order — a lot like email. Google Reader’s “All Items” is an example of this type of feed reader. All that work just to simulate email.

There’s a third kind that barely qualifies. It doesn’t track read vs unread, instead presenting the latest N headlines from the feed. It’s up to you to remember what you’ve read and haven’t read. Last time I checked (which was a long while ago) netvibes.com’s reader was like this. I’m sure it’s improved since then.

As you may have read, when I was at OSCON 08, a couple of guys are suggesting that large sites abandon this model in favor of a model more closely resembling email. I think their numbers were something like Site X asks flickr 7 million times for information about 54 thousand users when only 6 thousand total (not a subset of the 54k) updates were made. Obviously that many requests for so few updates is dumb.

I’m not trying to say that feeds are bad. I use them every day and love it. They’re especially great for embedding information from one site into another like my delicious bookmarks on my blog. That’s a use case other than the one I’m whining about, however.

The downside of having my blog software email a mailing list with every update is that the reader loses some anonymity. The upside is that an amazing amount of resources are saved. The feed bandwidth problem goes away. And you don’t have the 7 million for 6 thousand problem.

Of course no one will ever go for it. One, we’re way to far down this road. Two everyone hates email. Web-based bulletin board systems are a hugely popular and widespread testament to the fact that everyone hates email. At least syndication feeds have the whole “embed some part of my site in your site” thing going for them. Web BBSes have nothing at all going for them. A threaded email client (especially one with a kill list) wins over the web BBS 100% of the time and especially around security.

My confusion is renewed in seeing the rise of Twitter which takes the email-like instant messaging and gives it all of the scaling issues of syndication feeds and worse.

The only win I can give twitter or the web BBS is discover-ability, but both have the option of being closed to non-subscribers.

Part of the problem with email is that clients make it too difficult to sort an average user’s mail. I know I’ve talked to web BBS fans who cringe at the thought of a mailing list because they see it as fire-hose in their inbox. Maybe it’s not a difficulty thing. Maybe it’s just a task no one wants to do. If clients were smart about mailing-list headers….

Anyway. Blah blah blah. This post has been banging around my head for more than a year which is why it might seem tardy. I’m not saying anything new here, but I wanted to say it too, damn it.