Email is a spectacular failure

As the kids say “epic fail.” I’ll start with that so that hopefully my highly radioactive core of curmudgeon doesn’t shine through quite as brightly.

Now I’ll tell you a bunch of stuff you already know. Email works like this:

  1. A user fires up an email client and composes an email and clicks “Send.”
  2. The email client connects to its configured SMTP server and hands off the message.
  3. The email server looks up the MX (that’s Mail Transfer) DNS record(s) for the recipient(s), connects to the mail server(s) and hands off the message.
  4. The recipient(s) fire up their email clients and connect to their configured POP/IMAP/etc servers and retrieve the message.

It makes sense. It’s simple. It scales (just ask spammers!) I can only speculate why people don’t like it. Maybe email clients are terrible. Who am I kidding email clients are terrible. Even the best of them are awful for any number of reasons. But I’ll wager it’s more than that.

More stuff you already know: People blog, corporations blog, everybody blogs. Even if it’s not a blog, one still needs a way to notify one’s constituency that “I put something new on my website!!” No one wants to go visit a site over and over only to find nothing has changed. So feeds and feed readers were born. And to me, at least, this was a somewhat confusing turn of events.

Email, you see, is capable of sending HTML email and many, dare I say “most” email clients can render HTML email. If it’s not painfully obvious where I’m going here: it would seem quite logical to send an email to interested readers every time ones site is updated. Heck, it could even contain the full HTML of the update if you wanted. That way, blogs could work just like mailing lists of yore. Readers could comment on your blog entries via email. Things don’t work that way, though. They work like this:

  1. I write my snarky whiny blog entry and click Save in my blog software.
  2. The blog software converts my blog entry to at least one, but probably two different formats (RSS/ATOM.)
  3. Optionally, the blog software might have a second set of feeds for comments. Doing this same double-conversion on them as well.
  4. Should someone be interested, they use a feed reader to subscribe to one of these feeds.
  5. Their feed reader makes repeated requests to my blog software asking if anything is new.
  6. “Asking if anything is new” gives this solution too much credit. What it really does is ask about caching information (And that’s if the blog software has implemented caching.) and if the cache time-to-live has expired, it downloads the entire feed from the site, not just what’s new.
  7. Then it goes through the whole feed comparing it to what its already seen (a process that’s more difficult than you might think) and if something is, in fact, new it alerts the user that something is new.

Wow, that’s a lot.

The worst of these feed readers are really nothing more than glorified bookmarks. The names of the sites and maybe a favicon are presented in a list. The feeds that have unread entries have a bold title and a number of unread items next to the title. These are the worst because they go through all of the above nonsense for next to no value.

For readers like this, feeds could simply be an integer. If the last number you got from my site was 110 and now it’s 112, you know to show your user articles 111 and 112. In fact, this alone would be a huge improvement on the current system because feeds all have their own bandwidth. Since my feed doesn’t know when the last time you asked for it was, it can’t possibly know how many things have changed since you last asked. If I’ve updated 10 things, but my feed only contains 5, you’ve missed 5 updates.

Only slightly less awful than the bookmark style feed reader is modeled after, you guessed it, an email client. This feed reader combines feeds into a meta-feed that’s presented in some variety of chronological order — a lot like email. Google Reader’s “All Items” is an example of this type of feed reader. All that work just to simulate email.

There’s a third kind that barely qualifies. It doesn’t track read vs unread, instead presenting the latest N headlines from the feed. It’s up to you to remember what you’ve read and haven’t read. Last time I checked (which was a long while ago) netvibes.com’s reader was like this. I’m sure it’s improved since then.

As you may have read, when I was at OSCON 08, a couple of guys are suggesting that large sites abandon this model in favor of a model more closely resembling email. I think their numbers were something like Site X asks flickr 7 million times for information about 54 thousand users when only 6 thousand total (not a subset of the 54k) updates were made. Obviously that many requests for so few updates is dumb.

I’m not trying to say that feeds are bad. I use them every day and love it. They’re especially great for embedding information from one site into another like my delicious bookmarks on my blog. That’s a use case other than the one I’m whining about, however.

The downside of having my blog software email a mailing list with every update is that the reader loses some anonymity. The upside is that an amazing amount of resources are saved. The feed bandwidth problem goes away. And you don’t have the 7 million for 6 thousand problem.

Of course no one will ever go for it. One, we’re way to far down this road. Two everyone hates email. Web-based bulletin board systems are a hugely popular and widespread testament to the fact that everyone hates email. At least syndication feeds have the whole “embed some part of my site in your site” thing going for them. Web BBSes have nothing at all going for them. A threaded email client (especially one with a kill list) wins over the web BBS 100% of the time and especially around security.

My confusion is renewed in seeing the rise of Twitter which takes the email-like instant messaging and gives it all of the scaling issues of syndication feeds and worse.

The only win I can give twitter or the web BBS is discover-ability, but both have the option of being closed to non-subscribers.

Part of the problem with email is that clients make it too difficult to sort an average user’s mail. I know I’ve talked to web BBS fans who cringe at the thought of a mailing list because they see it as fire-hose in their inbox. Maybe it’s not a difficulty thing. Maybe it’s just a task no one wants to do. If clients were smart about mailing-list headers….

Anyway. Blah blah blah. This post has been banging around my head for more than a year which is why it might seem tardy. I’m not saying anything new here, but I wanted to say it too, damn it.

7 thoughts on “Email is a spectacular failure”

  1. I’m not sure I get it. POP3 or IMAP4 are both dependent on client software regularly checking a server for what should or should not be delivered to the client. The advantage for email is that an intermediary device caches the messages for the client, so it does not have to check each source. This is one of the reasons I both enjoy and loath Google Reader. It essentially acts like the equivalent to my mail server and holds the messages for me.

    Email is more push. Stuff comes at me without me having much control over the content I wish. RSS is more pull. I grab the content I wish and turn off the tap should I lose interest. As a user, I much prefer the pull approach.
    :)

  2. You’re right, of course, that email clients poll. The difference is that as a user, I’m subscribed to over 100 feeds. Were I using a desktop feed aggrigator, I’d be polling over 100 servers for information that may not have changed. If my aggrigator hasn’t implemented Etags (etc) correctly, I’ll be downloading the whole feeds over and over for nothing.

    The bigger problem is with sites like Google Reader hitting sites that have thousands of feeds (think Flickr). As I said in the post, were it email based, it would scale beautifully.

    If the whole world was using email lists, why would you feel you have less control over those subscriptions than you do over feed subscriptions? Just as easy to turn off either tap in my mind.

  3. Because all too often my email addresses get sold, traded, or whatever to become the recipient of unwanted junk mail. Turning off the tap for email is an unending war against The Flood. Once I use an email address, I have no control how another uses that address. Now that I have a hosting service where I can create whatever forwarding addresses I wish and drop them, that does serve to turn off the tap.

    Subscribing to an RSS feed takes 4 clicks. Subscribing to an email list with a forwarding email address would take way more clicks and keystrokes. Just simpler to use the broken tech.
    :)

  4. Now I have to disagree with you. Your email address is not a secret, sorry. You can’t really think that signing up for mailing lists would somehow give you more spam in your inbox do you?

    My google apps account gets about 40k (no typo) spams per month. I really doubt anything I do could make that worse. :)

    If subscribing to a mailing list was entering your address and clicking “go!” that’d be one ‘click’ :) Nothing stopping you from using single-use emails to see who is selling too. I do that for everything. When I signed up to see your jabber, the email I used was jabber at samrowe dot coo-coo-catchooooooo. No setup on my end other than saying “emails not destined for a specific account go to this inbox,” so I can make up addresses WILLY NILLY.

  5. Which is it? You said it is just as easy to turn off the tap for email. :)

    My forwarding email address = your single-use emails. My point is creating them is way more work that subscribing to an RSS feed.

    To subscribe to an email list: I have to create a new browser tab, click through some bookmark folders to my host’s login, type my credentials, click through to the email manager, type in a new email address to use, switch back to the list form, paste in the email address in the form, wait for the list confirmation, click on the link for the list confirmation, go to my email program, and create a filter. ~40 clicks or key strokes.

    To subscribe to an RSS feed: click the RSS icon in Firefox, click the Google Reader button, click the folder list, click the folder. 4 clicks.

  6. Creating my single use emails is thinking of a string to put before the at sign. The _only_ reason I make this is pure curiosity. As I said, your email address isn’t a secret. Your 40 clicks are a waste of your time.

    We’re way too deep in this hypothetical situation, but the reason that you perceive it to be difficult to sign up for an email list is because the tools you use have grown up around the broken system of feeds. If the world I described existed, your browser would make it as easy to subscribe to an email-based feed as an RSS/ATOM feed. It might even have support for making up the string I mentioned above or at the very least using the old-school + nomenclature that most mail systems support.

    And even if browsers didn’t have such support, but the blog engines (et al) supported email based feeds, all you’d need to do is type in your email address into my blog engine… like you already did to be able to post comments here. :)

  7. Feedburner does allow people to subscribe to my feed with an email address. In the six months I had the email subscription box available, I got 20 RSS subscribers and 0 email subscribers, so I dropped the box figuring no one would ever be interested. Little did I know there would be one person.
    :)

Leave a Reply