I’ve been seeing a lot of posts on the webz lately about how we can fix email. I have to say, I think it’s a bit short-sighted.

People are saying it has outgrown it’s original usage, or it contains bad error messages, or it’s not smart about the messages received.

These are very smart people, with real observations. The problem is, their observations are misplaced.

What email is

Email is a distributed, asynchronous messaging protocol. It does this well. It does this very well. So well, I’m getting a boner thinking about it. You send a message and it either goes where it’s supposed to, or you get an error message back. That’s it, that’s email. It’s simple. It works.

There’s no company controlling all messages and imposing their will on the ecosystem as a whole. There’s no single point of failure. It’s beautifully distributed and functions near-perfectly.

The problem

So why does it suck so much? It doesn’t. It’s awesome. The problem is the way people view it. Most of the perceived suckiness comes from its simplicity. It doesn’t manage your TODOs. It doesn’t have built-in calendaring. It doesn’t give you oral pleasure (personally I think this should be built into the spec though). So why don’t we build all these great things into it if they don’t exist? We could add TODOs and calendaring and dick-sucking to email!!

Because that’s a terrible idea. People are viewing email as an application; one that has limited features and needs to be extended so it supports more than just stupid messages.

This is wrong.

We need to view email as a framework, not an application. It is used for sending messages. That’s it. It does this reliably and predictably.

Replacing email with “smarter” features will inevitably leave people out. I understand the desire to have email just be one huge TODO list. But sometimes I just want to send a fucking message, not “make a TODO.” Boom, I just “broke” the new email.

Email works because it does nothing but messaging.

How do we fix it then?

We fix it by building smart clients. Let’s take a look at some of our email-smart friends.

Outlook has built-in calendaring. BUT WAIT!!!!! Calendaring isn’t part of email!!1 No, it’s not.

Gmail has labels. You can categorize your messages by using tags essentially. Also, based on usage patterns, Gmail can give weight to certain messages. That’s not part of email either!! No, my friend, it’s not.

Xobni also has built incredible contact-management and intelligence features on top of email. How do they know it’s time to take your daily shit before you do? Defecation scheduling is NOT part of the email spec!!

How have these companies made so much fucking money off of adding features to email that are not part of email?

It’s all in the client

They do it by building smart clients! As I said, you can send any message your heart desires using email. You can send JSON messages with a TODO payload and attach a plaintext fallback. If both clients understand it, then BAM! Instant TODO list protocol. There, you just fixed email. Easy, no? Why, with the right client, you could fly a fucking space shuttle with email. That’s right, dude, a fucking space shuttle.

If your client can create a message and send it, and the receiving client can decode it, you can build any protocol you want on top of email.

That’s it. Use your imaginations. I’ll say it one more time:

There’s nothing to fix

Repeat after me: “There’s nothing to fix!” If you have a problem with email, fork a client or build your own! Nobody’s stopping you from “fixing” email. Many people have made a lot of cash by “fixing” email.

We don’t have to sit in fluorescent-lit, university buildings deliberating for hours on end about how to change the spec to fit everyone’s new needs. We don’t need 100 stupid startups “disrupting” the “broken” email system with their new protocols, that will inevitably end up being  a proprietary, non-distributed, “ad hoc, informally-specified, bug-ridden, slow implementation of half of” the current email system.

Please don’t try to fix email, you’re just going to fuck it up!! Trust me, you can’t do any better. Instead, let’s build all of our awesome new features on top of an already beautifully-working system by making smarter clients.

Trackback

OMGOSH 69 comments

  1. Anonymous said

    Painfully obvious. Finally someone said this.

  2. Agreed 100%. I would also like to point out that at its core e-mail is one of the most reliable messaging systems that exists. MTAs will queue messages for a long time and retry transmission to multiple destinations in case of failure. Of course, spam filtering breaks the delivery guarantees, but nowadays spam filtering is actually pretty good.

  3. Agreed

  4. “Asynchronous messaging protocol”… On the dot.

  5. Have to first say, thank you so much for this. New clients can do so much for email than any stupid new protocol put out there. Take Shortmail for instance. It’s using the same old email protocols we all know and love, and adding a layer on top that says “No messages over 500 characters. That’s it. Oh, and you can choose to have email conversations be publicly viewable. Awesome. You could almost make your own little social network based on email. Stupid, stupid email.

    Lastly, thanks for the comedy. I will bookmark this site because I really like your style and you seem to talk some good sense.

  6. You have an error in the first part…sometimes you send an email, and the email doesn’t arrive. This is called google / hotmail spam filters.

  7. Finally!
    I would leave out the part about pimping the clients. Extra functionality is extra, you could as well have separate applications for your todo or dicks-sucking needs. Why implying they should be somehow connected to email?

    ‘Electronic Mail’, it’s not that hard to grasp.

  8. Glad you said this. This is exactly what we are doing with Boxure, making a better client, not changing Email as the protocol.

  9. [...] via Email is not broken: It’s a framework, not an application | kill the radio. [...]

  10. Email may be perfectly usable for you, but it still isn’t nearly as good as could be. Besides the usability problems, it has two significant technical flaws: imperfect SPAM filtering and insecurity.

    If all you ever need from email is a mechanism to insecurely and unreliabley deliver messages at low volume, than I agree with you. A lot of us expect more out of email than that.

  11. Agree on the overall message. The one part I’m interested in “fixing” is the interactions between client and server, in particular:

    * one protocol for sending and viewing, no “IMAP + SMTP”
    * reliable detection of copy, so a second client doesn’t have to download the same messages again.
    * better interoperability testing and removal of the “vast number of orthagonal CAPABILITIES which don’t reference each other” problem which means you can’t rely on anything but the basics when writing a new client that needs to talk to random servers – otherwise you wind up doing multiple, rarely tested, codepaths for each thing.

  12. Ronny Pfannschmidt said

    you are forgeting that while the delivery protocol (smtp) is a nice and functional imap and pop are quite a painfull mess

    there is a lot of breakage around those protocols

    its a falacy to declare the whole stack good, just cause one component is ok
    while he rest is quite a mess

  13. “We need to view email as a framework, not an application”

    While I agree with you to a point, unfortunately, the average email user considers email to be an application. As such, they expect it to “just work” Corporate users use Outlook voting buttons because they assume all recipients are using outlook and can vote using those buttons. All other clients are viewed as aberrations. Likewise, GUI based senders view purely text based mail clients to be “broken” because they can’t see the embedded images and rich text/HTML formatting in the message. Users view email the way they view their power utility. They don’t care how it works, they just want the light to come on when they flip the switch.

    Viewing email as a framework is fine for those who build the tools, but for the masses, it will always be an application.

  14. [...] read some surprising thoughts in a post Email is not broken: It's a framework, not an application. I'm not sure I understand any of the inference though. For starters, Email consists of several [...]

  15. I would argue that email actually is heavily broken. Because most clients totally suck at implementing it. In particular the enterprise IT part of the ecosystem. And because people totally suck at using it (often because of bad clients).

    Of course that’s not going to be fixed by replacing it with a new underspecified centralized mess (or whatever the same people who broke email in this way tend to dream up), but by _implementing_ _the_ _frickin_ _specs_.

    For example, email knows something called message-ids and references-/in-reply-to headers to enable clients to automatically sort messages into threads/to recognize when a message is a reply to something they have seen before. Only so that morons such as ebay or many ticket systems add some id to the subject or the body of the message and then urge you to keep that part of the message in the reply (essentially relying on full quoting, which is another way email is totally broken) so they can correlate the reply – instead of using the fully automatic mechanism built into email that does just what they need.

    OK, in part that’s probably because other email clients just don’t add those reference headers. Yet another way that email is broken because of sucky clients. But that doesn’t mean you have to break it further when working around it, of course.

    Also, there seems to be a new trend that enterprises send HTML-only mails. Often to then tell you inside that HTML where you can download a PDF invoice, say. Not only is that incompatible with plaintext-only clients, but it’s also a totally f*cked up direction of development as far as “the power of computers” is concerned. Many years ago I was expecting that in the future bills would come via email in machine-readable format. As a JSON document, say. Instead we are moving in a direction where people have to do more work to get their invoices into their hands, plus major overhead for managing all those PDF files that are then not automatically filed away as part of your mail folder. And all of that is not because email wasn’t capable of doing what is needed, it’s because people implement it in idiotic ways.

    Bottom line: before trying to reinvent email, read what’s in the specs, use some mailer that actually implements those features, and when you have understood how email actually works and what features it can provide, work on having those widely implemented to un-break email instead of fragmenting the communication infrastructure even further by introducing additional badly thought-out non-interoperable systems.

  16. @Flimm: What is insecure about email?

    And as for imperfect spam filtering: Erm … yeah, but that doesn’t have anything to do with it being email. You cannot perfectly filter spam, because spammers tend to not want to be filtered, hence they don’t add some this-is-spam-header to enable error-free filtering. If you want to have some publicly usable communication address, you cannot prevent with technical measures that people will send you unwanted communication. It’s the same with postal addresses, it’s the same with telephone numbers.

    As with all of these you can use secret addresses with known peers to establish a reliable and spam-free communication channel with them (actually it’s a lot easier, as every domain comes with an infinite number of addresses for you to use).

  17. Adam Dangoor said

    E-mail is lacking in one big way. Unlike Facebook Messages, and any messaging system of the future I guess, it treats you as an address. I am not an address, I have a name, school, location and face that you can associate with me. This is a fundamental issue that will, I guess, mean that e-mail won’t last too much longer.

  18. @Bron:

    @”* one protocol for sending and viewing, no “IMAP + SMTP””:

    Let’s invent a protocol called IMAPSMTP, it consists of IMAP and SMTP. In other words: a protocol is whatever you define as a protocol, and what you really want to be improved must be something else than the fact that IMAP and SMTP commonly are viewed as separate protocols. Maybe the configuration of the client?

    And in any case, you don’t need any “protocol” for viewing messages, hence it’s quite wise to have those separated. In my case, SMTP delivers directly to my workstation, where the MTA writes into the mailbox that my mail client reads. And technically, there is no reason why that’s not more common – in particular with IPv6, it would be perfectly reasonable for your computer to have a static address (in addition to possible dynamic ones) which your mail provider delivers your mail to.

    @”* reliable detection of copy, so a second client doesn’t have to download the same messages again.”

    I don’t quite get why a second client wouldn’t want to have any messages the first client got, and why the first client then doesn’t just remove the message from the server?!

    @”* better interoperability testing and removal of [redundant features]”

    I’m all for interoperability, but what redundant features? The email protocols don’t have much of that sort that I could think of.

  19. @Mike: Erm … well, that may be true, but there really is no other solution than to make people understand that it’s a framework.

    Of course, you can instead totally fragment the communication landscape by implementing each set of features as a clearly separate and distinct “application”, so that users understand why those “applications” are not interoperable. But other than that you haven’t gained anything. Quite to the contrary, you probably instead got rid of even basic interoperability that at least still exists with email.

  20. [...] on HN and various other places. Some of them are very business-oriented, and some of them are scathing rejections. But most of them miss the point. Email as a system is not broken, but we, through our email [...]

  21. You missed the point entirely. Wave may have been overly complicated, but their original statement about the problems with email was on the money…

    You can’t edit or retract an email.
    If you want to include someone in a conversation, you have to forward them a long, hard to read quoted mess.
    If everyone in a conversation doesn’t agree about who should be included, things break down, and it is very hard to tell that it is happening.

    It isn’t that email doesn’t have enough features, the problem is that email sucks to use. Forum style threaded discussions are just better. My inbox should be an aggregator for threaded discussions.

  22. @Nathan: You, too, are confusing your broken client with email in general.

    No, you don’t have to forward a hard to read quoted mess. Email has threading built-in, as I alluded to above. With a less sucky mail client, you just let the client find all messages belonging to the thread and then either bounce them (a function many sucky clients don’t even have) or attach them as message/rfc822 attachments to a new mail. The receiver then can read those mails just the same way you did, perfectly integrated into their mail processing.

    If your inbox is not an aggregator for threaded discussion, then that’s because your mail client sucks, not because the email specifications is lacking any necessary features.

    Exactly the lack of a uniform interface and oftentimes the lack of proper threading is why I despise web forums and very much prefer mailing lists.

    As for people not agreeing who to inclue in a discussion: Well, that’s only a problem with ad-hoc exchanges, and is inherent in it being ad-hoc/non-centralized (there just is no such thing as an “ad-hoc web forum”). For longer discussions/groups that need to discuss things regularly, mailing lists are the tool to use – if you add an archive, that also enables an easy way to get new people up to speed as to what is going on.

  23. Oh, and related to email, also NNTP/NNRP is not to be forgotten. That’s probably the closest you can get to “forums” with an easy to use interface. And you can use those protocols independently from Usenet. Newsclients did threading before the web was even invented, and to this day the interface is much more responsive and easy to use than any web forums.

  24. @Adam Dangoor: I think you are confusing two separate issues.

    Your facebook user account is as much an address as an email address is, in that it allows communication to be directed to you or proxies of yours. That address is needed in order to reach your school, location, and face, as you put it.

    And it’s not just that a facebook account is an address, too – it has to be. Without an address, communication is impossible.

    So, the fact that the “application” facebook allows certain information about you to be retrieved using the same address as the address that’s used for sending you messages is what you probably actually like there. But that’s not in any way a new invention: You can have a web page with information about you and with a contact form, or with an email link, without facebook. That web page then has a single address that can be used for accessing both of those services.

    Also, protocols such as XMPP/Jabber (primarily an instant messaging protocol) support publishing “profile information” under your XMPP address in addition to being able to receive messages there. And you can use the same address as your email address as your XMPP address if you want, so that effectively all those services are reachable at the same address – with the major advantage over facebook that there isn’t one single company controlling and potentially re-purposing all of it, And all of that existed before facebook, mind you.

  25. Andrew Lyon said

    @Anthony Bullard: Thanks, glad you like the style! It’s been getting some negative reviews, but as you probably realized already, I don’t care =).

    @Mike: re-read the post, not just the title. I’m saying email works great, the problem is the client. Yes, your typical user expects it to just work, but *this is a client issue*. The protocol works fine.

    @Adam Dangoor: Really? You are complaining because “I don’t want to be an address!! I’m a person!” Yes, we all get that we are sending *you* an email, not an address. An address is just an endpoint. That’s like saying houses are broken because people can find them.

    @foobar: Thanks for making all the responses so I don’t have to. Logged in this morning to find a ton of comments. Yes, email is a mess because of bad client implementations. So for the most part, I think this echoes what I’m trying to say. Would you agree the protocols, although a bit hairy sometimes, function very well for the most part?

    @Nathan: Use a forum. Email actually works great for the types of communication you’re talking about if people aren’t stupid. Email allows them the ability to be stupid though. What you’re talking about isn’t a protocol issue OR a client issue. It’s a user issue. Also, how would you *ever* be able to edit an email after it’s sent without having some sort of monolithic server control all emails everywhere? Exchange does this if you send an email to someone else on the same server, but we’d literally have to just have one exchange setup for the entire world. If you need to message people and require to edit that message, a forum will work fine.

    You cannot “fix” email without breaking it completely.

  26. Christina said

    I like the part about the boner.

  27. Email is not a framework, it’s a set of protocols. And those protocols *are* broken. Have you ever fucking tried setting up a working email server from scratch?

    Also, there is a limit to what you can do on the client side, especially without support of the underlying protocols. This is architecture 101. Just looks at how things developed. We use emails almost the exact same ways they’ve been used in the 70s. *Seventies*. That’s forty years ago. I don’t see any creative, wonderful systems build on to of email. Because the infrastructure is crap and people don’t want to invest in it. Because there are easier ways to do almost anything, despite email being nearly universal.

    And no, these problems are not inherent to *all* communication mediums. Spam is still far less of a problem on the web than it’s on email servers. And it’s fought with in much easier ways. Web servers are infinitely easier to set up too. I’m not saying web is ideal, but it’s an example of something widely used and much less broken.

  28. Aaron Peschel said

    The standard for valid email addresses needs to be fixed. It’s just way far too complex for an era where most addresses are of the form `[email protected]`. No sane person is going to have an email address like `”I dare you”.(to parse this ((((((hahaha)))))))”…..@”hint.”:”.you.probably.cannot+hahahaha@[IPv6:2001:db8:1ff::a0b:dbd0](good luck trying)` but such an address is technically a valid address.

    So you have to choose. Do you want a simple parser that will match 99.9% of email addresses from non-spammers, but is not actually a correct parser

    Or would you use a parser that is extremely complicated, much slower, and really only validates email addresses from spammers.

    Both suck in their own regard. Instead, just fix the specification on what a valid email address is. It is pointlessly complicated.

  29. our thoughts exactly… we’ve built a cool addressing scheme and email app where the web meets email. There’s so much potential for building on top of the email framework.

    examples:

    https://leemail.me/r/A0EBD (us)
    FollowupThen.com (no affiliation, but v. useful)

    Michael

  30. Thanks for taking the time to read my initial posterous post. I would like to note that while I talk about all of these issues being broken with email, my solution wasn’t to change the email protocol but rather to build a smarter client around the existing framework (as you suggested). That is why I mentioned roockit.com in the posterous post.

    ps – you can still just send a regular message with roockit. it is simply meant to help transform messages with tasks in them into something useful.

  31. @foobar
    You commented “Many years ago I was expecting that in the future bills would come via email in machine-readable format. As a JSON document, say.”. How do you see that working exactly? You also keep repeating that most problems are related to sucky email clients however make no note which email clients are not sucky and do most or all of what you mention. Be great if you could suggest alternatives.

  32. IMHO email will be “broken” as long as businesses continue to shove exchange+outlook down employees throats.

  33. @Andrew Lyon: Yeah, absolutely. Or at least all those people talking about “fixing email” are mostly without a clue as to where the real problems with email are, and are instead suggesting “fixes” to problems that have far superior solutions that have been standardized long ago and that have been implemented in many clients and/or servers, or to problems where after long discussions a long time ago it was concluded that there is no solution without inacceptable side effects (where the supposedly new solutions of course are among those that have been discussed back then).

    Implementing a full SMTP MTA or a full MIME stack probably isn’t particularly much fun, and the complexity that comes with it having grown over many decades may be a part of the reason that many client implementations are horribly broken, so certainly many things could be done better when starting from a clean slate. But if it is implemented as specified, it does work quite well, and the interoperability certainly is more valuable than having a slightly less baroque set of protocols.

  34. @Gambler: I for one haven’t only tried, I operate my own email server, yes. I don’t see any complexities there that are particularly due to the protocols?! Most MTAs in existence aren’t exactly meant for end-user installations, but rather for servicing whole companies or universities or ISPs, but I don’t see how the complexities that arise from that are inherent in the protocols?!

    As for “no change since the 70s”: As a matter of fact, a _lot_ has changed. It’s still all backward-compatible, yes, but hardly the same user experience. MIME didn’t exist back then, so you had to use tools such as uuencode for attachments, for example. But the really big change in my opinion is that new users at some point started full quoting. Back then, everyone who knew how to use email also knew how to make sensible use of quoting. At some point the accident happened that new users didn’t know what to do with the full quote that the MUA gives you as the starting point of your reply, and so they started writing their reply above the full quote, leaving the full quote sitting where it was. Somehow that became a meme, and now everyone (almost) is doing it. That step backwards is IMO the biggest problem email has nowadays, and a technical fix for this seems unlikely.

    Also, email doesn’t need “creative, wonderful systems”. Much the same way the postal system doesn’t need “creative, wonderful systems”. That is not to say that you cannot base new applications on the existing postal system, but it may just be good as-is.

    And in any case, there is no correlation between “old” and “crap”.

    As for spam: of course, it’s inherent in all communication mediums, as a matter of principle, no way around that.

    You know why spam is much less of a problem on the web? Well, for one, it isn’t. Ever heard of search engine spam? Maybe you don’t have to fight it there personally, but search engines certainly sink quite a bit of resources into keeping it away from you, and in some way that still is a societal cost that also affects you. On the other hand, most web-based communication systems aren’t as affected for the simple reason that they are so frickin fragmented. If there were a thousand diffent email systems that are not interoperable in any way, spam would be much less of a problem there as well. But that fragmentation comes at a huge cost in the form of a communication barrier. Any system that allows everyone to uniformly communicate with everyone else with minimal effort necessarily will also allow spammers to communicate to everyone else with minimal effort. It’s the network effect and the ease of use that makes a communication system attractive for normal users that also makes it attractive to spammers.

    After all, if somehow “the web” was magically protected against spam, then webmail systems also would have to be essentially spam free. The fact that stuff is transported via HTTP instead of SMTP has absolutely no effect on whether you are affected by spam. The effect you observe is simply due to many web applications being much more difficult to use and to interoperate.

    Also, just “setting up a web server” may be easier. But you may notice that a web server alone doesn’t give you more than a possibility to publish some files. Once you have to set up a database and install a web application it’s not that much easier anymore. Plus, most web applications are a pile of shit as far as security is concerned, whereas those days when sendmail was in wide use and a constand source of entertainment in the area of IT security are long gone, so you’ll have to invest much more effort to keep things secure in the long run with current web apps than with some of the better MTAs.

  35. @Aaron Peschel:

    Though some of the possible addresses are hilarious, I think you are wrong on two accounts:

    1. A correct parser is not “extremely complicated” and also not “much slower”. At least not in any meaningful way. It may be that some simpler parser is twice as fast, but given the microseconds either one takes for parsing an address, it’s really irrelevant. And even though the grammar is not regular (which is kindof hilarious), it’s not _that_ difficult to implement correctly either. And after all, you are not supposed to roll your own parsers for anything anyway. Every sensible highlevel programming language should have some ready-made library that you can use. That’s the advantage of having a standardized system: you need one lib per language (at most) to abstract away some of the peculiarities, and that usually gets implemented rather soon.

    2. You cannot fix a specification in that way, as unfortunate as that is. This certainly is one of the things one should have done differently (though that’s probably mostly in hindsight, after all there is a reason for why it is the way it is), but if you would make previously working addresses invalid, you are potentially breaking existing systems. And it is one of the big virtues of the IETF that they keep everything backward-compatible so as to never break existing systems unless it is absolutely unavoidable (such as for security reasons).

    All in all, the state of affairs isn’t perfect, but it is possible to implement it with limited effort, many implementations do exist, and interoperability and backward-compatibility really are far more important than some ease of implementation.

  36. I think a dick sucker would be a marvellous addition to the email protocol. I have been working on designing a dick sucking peripheral (attached by USB or Firewire) that can be controlled by a simple set of commands. The peripheral uses electronically sequenced elastomeric polymers which are controlled by a simple 8 bit micro-controller which receives its orders from the wire interface.

    So far, the peripheral understands a small repertoire of haptic opcodes, currently work is progressing on the swallow command.

  37. @PM:

    Well, I don’t have a finished spec in the drawer, and it’s actually more of an example of how we are moving more and more to a situation where computers are not used to automate routine stuff, but instead just simulate the analog world, and require the user to do the same (or even more) steps manually, just using the computer instead of an envelope and some sheets of paper, say.

    So, for example, companies could just send you their invoices in a standardized format (which could be based on JSON or XML or whatnot) that clearly marks up the items you are supposed to pay for, what each one costs, plus meta information such as bank account details and reference ids. This would be marked as some appropriate MIME content type, potentially as an alternative to some plaintext or HTML representation of the same information if you don’t have any software that can read the standardized format, so you can at least still “manually read” it.

    If you had any software installed that’s capable of interpreting this format, it could, for example, automatically import it into your banking software, where you could review the information and then with a single click authorize an appropriate wire transfer.

    Or it could automatically be imported into your household bookkeeping software so that you can easily compare, for example, yearly bills from your utility. Or you could then search in that software for stuff you bought earlier and possibly with a single click order the same thing again, with that also automatically authorizing the wire transfer that follows the order.

    Or you could have an additional format that online shops could use to let your package tracking software know when your order is on its way and at what URI of the delivery service further status information in yet another format can be obtained. Better yet you could automatically subscribe to status updates also being delivered by email.

    Well, as I said, I don’t have a finished concept, but the idea is to make the information you receive usable for your personal automation rather than just “electronic paper” that you still have to process manually. By using standardized formats, similar kinds of data would not sit in some shiny PDF file that’s only good for looking at or printing it, but instead you can have software that puts it all into a uniform interface.

    As for other mail clients: Well, I іntentionally didn’t want to make this a flamewar about what the only true mail client is, but given that you ask ;-)

    Well, I for one use mutt (www.mutt.org), which is extremely good at all the core email functionality (namely sorting, searching, threading, encryption, mailing list handling, general message processing (forwarding, bouncing, replying), identity management). The only thing that’s increasingly problematic is that some companies nowadays send HTML-only mails which mutt is not particularly well-suited for (you can handle it, but in contrast to everything else it’s not particularly comfortable).

    Other than that, I haven’t used other mailers much in a long time, so I can’t really comment much, except that in general the shinier ones usually lack in technical quality, with outlook (express)/windows mail bordering on unusable. If anyone else here knows some GUI mail program that is of comparable technical quality and featureset to mutt, I’d be interested in knowing about it :-)

  38. @foobar
    Interesting points you raised about a common standard format although how would you for example translate a format based on JSON or XML to something that is presentable. For example, most users are comfortable with using PDF documents to view bills, statements, etc. How would you translate the standard format to one that can be represented in an application like Adobe without user intervention? I don’t see that being the responsibility of the email client and doubt many enterprises such as Adobe would be willing to change what they do unless there is a demand for it and they can monetize it.

    Considering that most users want something shiny and beautiful looking as a client, how do you untrain the years of using say Outlook for something like Mutt. Haven’t used it myself so can’t comment on how easy it would be but unlike most people am willing to change for something better.

    Lastly be keen to discuss some of the points you raised offline. Any chance of me dropping you an email or you dropping me an email? Not sure whether we can post email addresses here.

  39. @PM:

    Well, the point is essentially that there actually is no need for it to be in a “presentable format”. That’s just what approximates paper bills as closely as possible, thus inheriting most of the problems (or let’s say lack of features) that paper bills have.

    But the point of an invoice, after all, is not to make you have a piece of (digital) paper in some fancy design where there are some numbers on there somewhere. The point of an invoice is to enable you to check you are paying for what you ordered/used, and possibly to make a payment. Hence you should not receive invoices as “digital paper” that you can then view using a “digital paper viewer” in order to manually extract from it the information you need (such es bank account details), but rather in a format that allows some specialized bookkeeping-and-banking-software (for example) on your computer to understand what it’s all about.

    Of course that software then may also be able to show you the information of that invoice, but the specific way any piece of software that could be used for the purpose would present an invoice to you really is a matter of that software’s user interface. Probably some spreadsheet-like interface would be useful, where you could directly sort the data and maybe do some simple arithmetic on it – but most importantly: You should be able to just copy and paste items from that view into a real spreadsheet application if you wanted to so some more in-depth analysis. Next to the item list there probably would be some button that lets you authorize the payment.

    When I receive a PDF invoice, in contrast, I usually end up looking for the bank account details and the amount to pay, selecting and copying and pasting those manually, one piece at a time, from the PDF into my homebanking application. That’s not much more convenient than when I had to type those in from a piece of paper, or earlier yet when I had to copy them from one piece of paper to another manually, it’s just using a computer to manipulate a virtual piece of paper, rather than making the computer understand what it’s all about and making it perform all the steps automatically, except for what absolutely necessarily needs to be done manually – namely, reviewing and authorizing the payment.

    In a way, that’s the essence of what is going horribly wrong with IT and the internet in recent years: Your PC is increasingly degraded to a virtual paper handling station instead of being a universal computer that does actual information processing for you, with privacy and in your interest. And part of the reason for that is that it’s difficult to monetize. There isn’t much money to be made in providing end users with the power to be autonomous, but a lot in locking them into proprietary solutions.

    As for “untraining users”: Well, no clue? Not at all, I suppose? I guess it would be easier to make at least some of the shinier clients get technically better – but even that I don’t seriously expect to happen for most of them. It really is difficult to say what could be done to get things into a better shape overall. I guess getting rid of outlook, exchange, and lotus notes would go a long way. But doesn’t sound particularly realistic either ;-)

    Generally I prefer to keep such discussions public, in the interest of anyone who might find them useful, as long as it’s not about personal stuff. Even if blog comments aren’t exactly the easiest to use communication medium either – I’ll reconsider if it gets too tedious ;-)

  40. Foobar:

    If everyone uses the same client, and everyone is part of the conversation from the start, and nobody needs to edit anything they send, MAYBE you could say it is a client issue. Unfortunately the previous situation is a fantasy. People don’t all use the same client, people join conversations late, and people need to edit comments. Maybe if we lived in a fantasy world where all the clients behaved well together and when people joined a conversation late clients could properly thread and manage everything, it would be different (that still doesn’t manage editing though).

  41. Do you want to know why these smarter clients aren’t materializing? Because the email protocols are an insane pile of forty-year-old hacks that get in the way of writing them.

    The basic architecture is brilliant and should be preserved. But the details of it—the line length limits, the weird escaping requirements, the use of termination tokens instead of length fields, the bag-on-the-side nature of MIME, the cryptic bounce messages that aren’t readable by machines or people, the lack of message encryption or authentication, the entirety of IMAP—these things are awful, and the world would be a better place if they were fixed.

    That world would be one where you could write your own client that forwards whole threads or attaches and detaches todo items or pays your bills for you. But in this world it’s way too hard just to get your one pet feature.

    (And of course you could have message editing and retraction with a decentralized email model. Just send the new message with headers indicating it should replace the old one. Technically inclined users could presumably circumvent it, but you can only do so much.)

  42. @Nathan:

    Well, editing in my book is an anti-feature. Just send a new message.

    Other than that: Well, yeah, but that was essentially my point: You won’t fix this by having everyone implement shitty versions of an email replacement, you will need good implementations. And if you have good implementations, you can just as well keep email.

    Though it’s absolutely not needed that everyone uses the same clients, there are plenty of decent ones, as far as sensible interoperability is concerned. And that’s not just a theoretical standpoint – there are mailing lists such as the linux kernel mailing list (the central point of coordination of linux kernel development) with some 6500 subscribers that distributes around 400 messages every day, and you can bet that the subscribers are using a wide variety of mail clients without any major problems. And you can bet that those people who are communicating through that list do know about the existence of web forums and would be perfectly capable of operating one if that somehow simplified the handling of those tons of messages. There just are some mail clients that are particularly widely used amongst end users that suck horribly.

  43. I very much agree that the perceived problems with “email” in almost all cases are in fact the result of unfortunate design decisions in specific products. One particular pet hate of mine is Microsoft’s cluster of misfeatures known as Exchange and Outlook. A little more than a year back I spewed forth a ranting blog post which is somewhat in the same track as yours, perhaps too wordy but heartfelt to the last bit – http://bsdly.blogspot.ca/2011/02/problem-isnt-email-its-microsoft.html

    - Peter

  44. [...] собой нашлись и защитники электронной почты. Email is not broken: It’s a framework, not an application — руки прочь от  электронной почты. Она реализует [...]

  45. If it ain’t broke, don’t fix it.

  46. @foobar

    I agree that any data delivered to a user should be in a format that can be recycled easily and efficiently although disagree on its presentation not being vital unless this can be articulated in some other way. For example an organization I work for sends monthly bills. As part of this bill, it includes usage data, graphs of peaks and troughs and to end it all, it includes services and products that the organization attempts to up sell if you are a customer that fits criterion x, y and z. So if we take the proposal of having the information in say JSON and XML, how do you see the rest of the data being presented? Often sensibility escapes productivity especially when it comes to marketeers. Sometimes they are mutually exclusive. The only solution I see is that an organization offers the ability to download the data in JSON or XML OR as a document such as PDF. Not sure whether that is viable as well at least for an organization.

  47. @foobar

    Know you said that mutt was an excellent email client however is there a list of features you see being anti-productive vs a list of features that deliver what is considered best of the breed? How would these lists complement organization requirements?

  48. Next time, leave out the fellatio.

  49. @PM:

    Well, obviously, an email can contain more than just a single attachment, so you still could add additional text or further attachments, much the same way you can put some flyer into the same envelope as a bill. That’s just not functionally part of the bill itself, and so it shouldn’t be confused with it.

    One also could in principle send the bill in multiple formats – MIME even has a mechanism for that (the multipart/alternative content type) which allows you to have multiple representations of the same information among which the receiver can select the best one that it can interpret.

    Also, one could argue that sending graphs also is the wrong approach. To make use of computers’ capabilities, one should instead get the raw data, possibly with some instructions for some default display included, similar to a spreadsheet document with some embedded graphing component, so that the graphing is done on the receiver’s computer, and the receiver can change the display if they want to see some other aspect, or they can combine the data with data from other sources.

    However: Never, ever even think about offering such data for download ;-)

    Making bills available for download is comparable to sending out letters that tell the recipient to pick up a sheet of paper from the nearest post office, and please bring with you some form of id, and then handing out square-meter pieces of paper.

    Making data available for download may be sensible for exceedingly large amounts of data, but other than that, just send it directly, data transfer for the most part doesn’t cost anything anymore, and it’s just a lot easier if all the mail one receives is managed by one piece of software, and you don’t have to also keep track of a bunch of files that in a way are a kind of “mail” as well but are just lying around in your filesystem (analogous to square-meter pieces of paper that are slightly difficult to handle as part of your normal document management).

    An even better solution for large(r) amounts of data may be to just allow the user/customer to choose which data they want to receive. That’s a one-time choice to make, and after that stuff just gets delivered to where it’s easiest to handle.

    Oh, and of course nothing wrong with having copies of the data available for download lateron, it’s just a bad idea for the primary delivery mechanism.

    As for selecting a MUA: Well, that’s really a difficult question. As has been mentioned before, email is broken in a way due to certain widely-used clients and a lack of training of many users. If you often have to interact with people (incompetently) using one of the particularly bad mailers, then some form of compatibility with their brokenness probably would be a high priority, as that tends to cause friction with mail clients that can be used in a highly efficient manner in environments such as the linux kernel ml, where broken mail clients are a minority.

    Essentially, much of this is a network effect: If you primarily interact with people competently using a good mail client, you can gain a lot by also using one. If you have to primarily communicate with people who do not, it may even be that things get worse for you. It’s kindof like if you bought a fully automatic scanner that you could feed all your incoming snail mail into in order for it to be scanned into your electronic document management system. This can hugely boost efficiency. Unless, of course, your customers always send you their notices of cancellation on various large pieces of cardboard with various flowers and other decoration glued onto it, and written with fingerpaint. If that’s the bulk of your incoming mail, there is just no hope that better technology will help you much.

    As far as core mail processing functionality is concerned, I guess this list from an earlier post sums it up pretty well:

    (namely sorting, searching, threading, encryption, mailing list handling, general message processing (forwarding, bouncing, replying), identity management)

    For longer mail exchanges, proper threading support is really crucial, and one of the things bad clients often are particularly bad at.

    Anti-features as far as efficient mail handling is concerned: HTML mail, automatic reflowing of quoted text, forcing a full quote that you cannot insert your reply into, potentially the use of a proportional font (with a proportional font you cannot do any alignment of text using spaces, as it would only work out by chance if the recipient happens to use the same font, whereas everyone has a fixed-width font available).

    http://www.guckes.net/mail/edit.html seems to be a decent summary of the most common bad habits when writing emails, so that should give some idea as to what kind of usage patterns a good mail client should support as well.

  50. @foobar

    I don’t quite agree on sending multiple attachments when a message can be delivered using a single medium of communication i.e. a single attachment. Personally I think, we need to step back and look at who the majority of users are and how competent they are. For example, out of all the customers an organization may deal with how many of them are savvy enough to want to slice and dice data compared to those who simply want to know what their usage is, how it compared to last month, what promotions may be on so they can save money, etc. As an example, what compromise would you see with the demands of what marketeers deem to see as user experience fulfillment compared to what you consider what an email client should be taking into account that most users fall under either using a client such as Outlook or a web based client such as GMail, Hotmail, Yahoo, etc.

  51. @PM:

    Any _reason_ for not agreeing? It seems rather arbitrary to me to reject something just because it consists of multiple mime parts. After all, you seem to be happy with sending a mail consisting of both a text body and a PDF attachment. Technically, there is no reason for doing that, and if you ask me, it’s often rather idiotic: It is perfectly possible to put the information of an invoice into the body of an email, and the only reason why PDF attachments are common instead, is that it emulates paper. Except I don’t want to receive virtual paper, I want to receive an invoice. And email is not a paper transport. Much the same way that letter envelopes are not a transport for clay tablets.

    The remainder of what you write essentially gets it all backward, IMO. Of course, they are “not savvy enough”, because they aren’t even aware of the possibility. And that won’t change unless someone starts doing it. Technical progress does not happen by waiting for end users to provide the specification. It happens by providing opportunities for them to learn a new tool/approach/whatever. The internet didn’t get into the homes of people because they 30 years ago demanded a packet-switched network with a hypertext service, both of which noone would have had a clue what that even was. Still, nowadays almost everyone has learned at least some basic skills necessary to interact with said hypertext system, namely the web.

    Hence the approach of staying backward compatible by providing both the more advanced representation and some common denominator that everyone can make use of, albeit without the advanced features.

    There is BTW another way in which you potentially are already using the same mechanism, just without being aware of it: If you write an HTML email with one of the common GUI mail clients, most likely it will automatically create a plaintext representation of the same text, and the mail you send then contains both the HTML and the plaintext version. If the mail client of the recipient is capable of rendering HTML, it will do so, and there will be no sign of there being a second representation of the same information contained in the same email. If the receiver’s mail client only can display plain text, it will display only the plaintext version.

  52. @foobar
    I should have been clear with my disagreement. I do not condone the sending of PDF files in an email that should clearly display the amount that is payable as an example. I completely agree that the only reason PDF files are there are to emulate paper. I disagree with the fact that we should be sending any attachments at all. In my opinion we should clearly state the objective of the email in plain text and THEN offer the user options in which to download the information be in a dynamically generated PDF, XML or JSON.

    I do standby my comment that many users are not savvy enough to understand the difference. For example we may approach the issue of emails from a technical perspective i.e. how best to deliver data however to moms and pops they just want it in a format they can either store digitally be it a PDF document or a format that integrates with their accounting software without they having to worry about how to integrate the solutions. Some users I deal with simply want to be able to print their documents. No more no less.

    I do agree that things only change when someone takes charge and it then forces those with minimal skills to adapt. A good example is tradesmen (well at least here) who were against the notion of “I need a website so that people know about the services I offer. If they want something they can call me.”

    However considering some were early adopters and that they clearly benefited from being the ones offering something people were seeking i.e. information it forced those left behind to change and adapt or slowly dwindle away.

  53. I suspect most readers will object the view of email being a todo list or maybe not – http://paulgraham.com/ambitious.html

  54. [...] I recently read a great blog post about how email itself should be viewed as a framework, and not as “broken”, simply because it’s a 30 year old technology that lacks some of the features of modern collaboration tools such as Exchange. As an Email Admin myself, I found the article to hit the nail right on the head. However, there are a few problems, that aren’t deal breakers, but are hurdles that would need to be overcome. You can find the original article over on Kill The Radio [...]

  55. @PM:

    I think we might be thinking about something slightly differently when using the term “attachment”. When I use the term, I simply mean the technical mechanism that’s commonly used nowadays for packaging multiple pieces of data into one email message, namely MIME multipart entities. This has nothing to do with how those pieces then are processed by the receiving client or how they are displayed to the user – that’s what I meant to demonstrate with the HTML/plaintext alternative example, where most users aren’t even aware that there are two copies of the information in the email.

    So, the idea isn’t to flood the user with tons of “files” that they have no clue what to do with, or anything of that sort, but rather to enrich the email with machine-readable information as to what the message is all about, so that mail clients capable of understanding this information can provide advanced functionality – MIME “attachments” simply would be an obvious existing technical mechanism that could be used for implementing this.

    The user experience would then work roughly like this: If you are using some old mail client, it should just display you a text body that tells you what you have to pay to whom for what, plus maybe some advertisement. If you do have an advanced client, though, it may offer some button to import the invoice into your accounting software. Maybe, with the use of electronic signatures, it could even do the import automatically. The accounting software in turn may be just a simple online banking client that just allows you to view the list of items and initiate the payment. Or it could be some advanced package that allows you to do complex statistical analysis on the data.

    The point is that the email contains the information in both human-readable form for backward compatibility and in machine-readable form for advanced use for anyone who has the means to make use of it, and the degree of sophistication with which the data is used by the recipient is determined by the software they choose to use. Moms and pops could just print out the text body and use it the way they are used to. I possibly would configure my system to check whether the invoice is what is to be expected for regular stuff and if so automatically execute the payment, only asking me for review if things seem wrong.

    Integrating the solutions obviously would be the task of whoever provides the environment. Operating system vendors/projects, IT departments, other independent software vendors/projects. Some kind of standards body of course would be useful for specifying the technical details of the formats so that the different implementations can interoperate. I guess the W3C would be a good place for this kind of stuff (and I wouldn’t even be surprised if someone there is already working on something like that, maybe some standard that noone implements even already exists :-).

    As for separately downloading additional data: Well, I think there is just no reason for doing that. It’s just additional manual steps, while the whole point of machine-readable information is to make the computer do routine stuff for you. And there is no advantage to it whatsoever, it just forces you to manually obtain an additional file, which logically belongs to the mail you received, but you have to store/archive it somewhere else, and usually obtaining it isn’t even as simple as clicking a link as you first have to log in to some website for it (which cannot easily be avoided, as you have to make sure noone but your customer has access to their information).

    Also, having all the information archived in one place might be useful even for people who use software that has no clue as to how to make use of the additional data. For example, if you decide lateron to migrate to software with more features, or maybe you just upgrade your software stack, you could retroactively import all the information that has accumulated to make use of it for analysis.

    Really, that isn’t all that different from how many people handle paper bills: Many companies will send you multiple pages of detailed information with their bills. Many people don’t actually read that, but just file it away. To potentially look into it later if they become aware by other means that some company has been billing stuff incorrectly, for example. And so far I haven’t seen any company that tells you in their bills that you can pick up the pages with the detailed information from their headquarters if you want to read them.

    There is only one reason why you shouldn’t just send all the information in one package: If delivering it becomes expensive. That’s essentially never a problem for the sender (internet bandwidth in the data center is dirt cheap), but recipients may be on some relatively expensive mobile network. But practically in that regard the use of tons of images in HTML mails, which is already common, is much more of a problem than providing raw machine-readable data, which almost never exceeds a few tens of kilobytes even for some form of detailed usage records, which shouldn’t cost you more than a few cents to receive even with the most expensive plans.

  56. Oh, and yes, I do use mutt as my todo list – though with some macros and stuff, and additional software that just sends me mails for regular tasks, so I do use the essentially same interface for my todo list as I do for email, but I think that’s really more a matter of the specific software and how you can extend/misuse ;-) it than a general feature of email. And whether this works out highly depends on how you organize your work and what kind of features you need in your todo list, obviously.

  57. @foobar

    I have to concede to your last post about MIME multiparts entities. Only if everyone I had conversations with was this interesting.

    As on the point of a todo list, to be honest I don’t quite understand the context of one in the world of emails. An email is simply an extension of a conversation you could be having over the phone or in person. Be keen to understand how a todo list fits into the entire realm.

  58. @foobar

    I would like to have a discussion on a different topic i.e. standards so would it be possible for me to drop you an email or for you to email me at peanuts\nospace\monkeys\@gmail\.com?

  59. [...] a constant flow of “email is/is not broken” articles across the internet, but most of them miss the point. Email as a system is not [...]

  60. [...] a constant flow of “email is/not broken” articles across the internet, but most of them miss the point. Email as a system is not [...]

  61. [...] admin on Apr.19, 2012, under Technology There's a constant flow of "email is/is not broken" articles across the internet, but most of them miss the point. Email as a system is not broken, [...]

  62. [...] fanget altså denne artikkelen min interesse: There’s a constant flow of «email is/is not broken» articles across the internet, but most of them miss the point. Email as a system is not [...]

  63. Agreed – it’s not the system, which works incredibly well. It’s the client. The original system was designed to send plain text between sender and recipient. But now there all types of senders and all types of recipients. This requires a more intuitive interface to prioritize the conversation (and at the end of the day, it’s always about the sender and recipient, not the category of message).

    Lots of folks are trying to solve this from the power user, personal communication side of things. We’re doing it from the average user, brand side of things. http://www.philterit.com. Forget the bells and whistles, just make the interface simple, intuitive and add pretty pictures. Let us know what you think @alevine0

  64. [...] find a low rumbling in the hacker community about how email is broken 1, 2 (or not broken 3, 4). Email is still the main communication medium on the planet besides Facebook, SMS, and the [...]

  65. [...] conclusion of this great post is what all people thinking email is dead need to read. I’m looking at all those teenagers [...]

  66. [...] hatte die Tage schon drauf hingewiesen: Email is not broken. "Email is a distributed, asynchronous messaging protocol. It does this well. [...] There’s no [...]

  67. Подсказали что нам поможет флуконазол виды помогает ли он?

  68. @Adam Dangoor

    Fuck off you ridiculous hipster. I’ve used Facebook messaging, and seeing someone’s face when you message them is a complete non-feature. Sure, it’s seems great if you’re the kind of person who reads too much Seth Godin, but in reality it’s utterly useless.

    Also, many mail clients already have this (optional!) feature. GMail uses your profile info if you allow it to and many services use Gravatar if you have an account.

    90% of the time, I’m not emailing people who I want to see my personal details. Facebook messaging is brain-damaged. If I want to see someone’s face I’ll meet them in person.