51 thoughts on “Facebook Pokes XMPP. MSN, Yahoo & AIM Better Watch Out”

  1. This might turn out to be a bad thing for facebook as people will no longer need to login to facebook to chat with friends. The point of having facebook vs. other independent services is that, it gives users single place where they can go and socialize.

  2. I love the fact that push-based/pubsub protocols are coming to the forefront. I must say it’s somewhat baffling XMPP has been the leader of the pack. Their strength is a strong developer community starting from the old Jabber days. From a protocol design perspective it’s a bit of a disaster, though. To start with, why is it XML? I understand text-based protocols are easier to understand and all of that, but we’re talking about a protocol that’s specifically used to transport small messages. In that context, header size matters a whole lot. With XMPP, it’s common for the headers to be bigger than the message.

    HTTP is also text-based, but it’s a completely different story because HTTP message bodies tend to be far larger than the header size. XMPP piles on top of it XML — not only is it text based, but it chooses to pile a whole bunch of XML on top of that. Yikes!

    I really wish developers were more versed in writing binary protocols. It comes back to the age-old lesson: usability always wins. Even on that score, though, PubSubHubbub is even simpler, less, verbose, not a new protocol (all HTTP), etc etc. See:


    I find this all a bit disheartening for some reason, even though it’s overall good news!

      1. Agreed — PubSubHubbub will hopefully do to XMPP what REST did to SOAP. I don’t think XMPP’s quite as bad as SOAP is (better in some respects, worse in some respects), but it’s in the ballpark.

    1. My feelings for you are complicated. You make a couple of good, or at least argument-worthy, points (XML+HTTP might be overkill, there are better text-based protocols than XMPP) and yet, you manage to get a really, really bad one in there (that we should start writing binary protocols). Not sure whether to consider your arguments or whether to repeatedly smash my head against my keyboard and monitor while screaming “BINARY PROTOCOLS”. Hmmm…. 😉

      1. My feelings for you are not complicated. You seem to be attacking my position that binary protocols make more sense when protocol payloads are small. You’re argument against that position seems to be something about smashing one’s head against a keyboard and that my point is “a really, really bad one.” Strange argument.

        Note that all the really slick, super efficient new protocols out there are binary: avro, protocolbuffers, thrift, etc.

        If you have an articulate response to my point, I’d love to discuss it. My perspective could easily be wrong. I have a lot of experience with protocol design, however, and that’s my feeling about XMPP. Not sure what yours is.

      2. Woohoo, we’re nested too deeply here for me to reply directly to you. Cool.

        > Strange argument.

        You’re a smart person, and I suspect you know better than to think that I was suggesting that “this makes me want to smash my head on my keyboard” was actually an argument against your point. It wasn’t. That’s just how I feel about binary protocols, or at least the instinctive desire to resort to them.

        > “Note that all the really slick, super efficient new protocols out there are binary: avro, protocolbuffers, thrift, etc.”

        Well-designed binary protocols are more efficient, at least for that very narrow definition of “efficient” whereby bandwidth usage is the only consideration. I’ll happily give you that point, because I’d never argue against it.

        My point, if I have one, is that efficiency is overrated. Yup, you might be talking a kilobyte or maybe even two of protocol overhead per message for an application like this. That is surely the end of the world; the two people who are still using 56k modems might, just might, notice messages taking a fraction of a second longer to send, if those gains aren’t swamped by latency anyway.

        > You seem to be attacking my position that binary protocols make more sense when protocol payloads are small.

        Well, payload *sizes* aren’t important (extreme example: how much more efficient would IRC have been if it had used a binary protocol? How much less robust would the implementations have been if it did?). There’s a case to be made for either text or binary protocols regardless of payload size. I do insist that someone demonstrates that a text-based protocol, even a bloated one like XMPP, causes *noticeable* inefficiencies, in the real world, that outweigh the many, many benefits of protocol transparency, before we resort to using binary protocols.

        I probably disagree with you less than it might seem. 🙂 Even though XMPP makes me twitchy too, it still seems like a good choice in this case, if only because it’s so widely implemented in IM clients…

      3. A couple quick points:

        1) Binary protocols don’t only save bandwidth. They typically also save CPU and memory.

        2) These protocols are rapidly becoming a part of the core Internet fabric. As such, they’re starting to see massive volumes of data passing through them, and this going to grow as companies like Facebook make greater use of them. As such, performance is a vital concern. Better protocol design would save everyone a great deal of time and money, with ultimately better performance for the user.

        3) Binary protocols are not more error prone. You could certainly make an argument either way, but binary protocols tend to be more explicit in what they allow for each byte. To me, they’re less error prone.

        4) The only upside I see to XMPP being text based is that more people can write XMPP applications. When it comes to XMPP servers processing the kinds of volume Facebook and Twitter and processing, though, you’re talking about highly skilled engineers that could just as easily write binary protocols as they could text-based protocols. If they’re not good enough to do that, we don’t want them writing our XMPP servers. On the client side, I think what you’re saying makes sense. It should be easy for people to write clients, and not all of them should necessarily be comfortable with writing binary protocols. So it’s certainly a tradeoff. To me, though, we should just go all the way and design the best Internet possible now.

      4. 1) Citation needed. Prove that it’s an issue on modern computers. I’m sure Facebook and Google will give you a call when they run out of CPU cycles.

        2) See “citation needed” above. Like anyone’s going to notice a few kilobytes of XMPP transactions above the shitstorm of multi-gigabyte HTTP and BitTorrent traffic that most . Show to me that it’s a real-world issue. Then we can talk.

        3) That’s where you show me that you’re an excellent protocol *designer*. You work out the rest. 😉

        4) Don’t sniff at the value of anyone being able to implement XMPP. Yes, if you’re working at Facebook or Twitter, you could probably make some really compact binary protocol and have all your engineers understand it etc etc. Great stuff. Not get that out into real-world implementations, built by amateurs who have better things to do, and that won’t count for shit.

      5. I really don’t mean to show any disrespect to the hard work of the XMPP protocol designers. Like I said, the addition of XMPP is a huge improvement over the existing Internet and opens up many exciting possibilities. I do think it could be better and that some of those initial design decisions could have been better, however. Again, I could be convinced otherwise and admit to being frequently wrong, but that’s my view.

        As far as a “Citation needed” about modern computers running out of resources as a result of these issues, I think that’s a non-starter. For the volumes of data we’re talking about, you’re clearly in the realm of running many many boxes to get the job done. If you have to run fewer of those boxes to get the job done with a more efficient protocol (again, less CPU, memory, bandwidth), you just save money, electricity, space, etc. So I guess I’ll just cite common sense, which to me is always the best citation!

    2. OMG, will I really receive my messages a millisecond later because they send my 100 bytes of XML header?

      SCNR… 🙂

    3. “With XMPP, it’s common for the headers to be bigger than the message. ”

      Last time I checked, XMPP didn’t have any headers in the HTTP sense of the term… Are we talking about the same XMPP ?

      “HTTP is also text-based, but it’s a completely different story because HTTP message bodies tend to be far larger than the header size.”

      Guess what ? It takes quite some time to setup a TCP connection. And I guess you already know that the connection gets teared down and re setup many times in an HTTP based architecture.

      “XMPP piles on top of it XML — not only is it text based, but it chooses to pile a whole bunch of XML on top of that. Yikes!”

      Did you get the XML flu or something like that ?

      “I really wish developers were more versed in writing binary protocols. It comes back to the age-old lesson: usability always wins. Even on that score, though, PubSubHubbub is even simpler, less, verbose, not a new protocol (all HTTP), etc etc.”

      This sentence doesn’t really make much sense does it ? You start by praising binary protocols, and then start talking about PubSubHubbub which let me remind you is not binary, but rather a set of hacks built on top of the venerable Text based HTTP protocol…

      Please can you quit talking nonsense ?

      1. Thanks for letting me know HTTP is text-based and what I’m referring to as headers are really more properly called wrappers in XMPP land. That added a lot to the conversation — really getting to the heart of the matter. It’s also useful to know you’re unaware of persistent HTTP connections. Thanks for all of that, really.

      2. Ali and Lewis Collard,

        Text based protocols may seem easy but often leads to unnecessary complexities. For the sake of human readability, protocols that start out simple at first eventually gets bloated and overly complex. Just look at the WS-* protocol stack if you want an example. And not to mention characters being garbled because the creator of some homepages didn’t bother to define the charset.

        While we have problems like endianness in binary communications, text encoding is no simpler and can also be ambiguous which causes hidden headaches and sometimes nasty security holes too. I guess you must prefer something which seems easy at first but is hard to validate until runtime. Script kiddie mentality.

  3. Everyone who uses Facebook chat in Pidgin, Adium or Trillium will know exactly how great news this is. Facebook has had issues with connections and multiple logins. (Running an IM client logged into chat, while trying to access Facebook.com in a browser). Basically, you’ve been plagued with disconnection problems, missing messages, duplicate messages, and errors while sending.
    Facebook chat moving to XMPP is going to be AWESOME. All of our favorite universal chat apps will probably run flawlessly now. This is great news.

  4. A few points about XMPP:

    1. We didn’t have protocol buffers and the like back in 1999. Feel free to design a new binary collaboration protocol from the ground up using a non-XML technology if you’d like. 🙂

    2. XMPP messages are not on the order of a kilobyte, think more like 100 bytes or less! We have deliberately kept the footprint light.

    3. The major bandwidth hog in an IM system is presence, not messaging. We have kept presence very small. To save bandwidth on mobile devices and such, don’t share presence.

    4. XMPP has native compression (via TLS or at the application layer) and that is very efficient because we use long-lived XML streams.

    I seriously question the charge that XMPP is bloated. Yes, it is bigger than a purely binary protocol might have been, but there are legitimate design tradeoffs of size vs. ease of understanding (and ease of extension). Anyone who designs that binary collaboration protocol to rule the world will run into the same issues. TANSTAAFL. 🙂

    1. Peter, I’m so disappointed in your shortsightedness.

      Adam Fisk is complaining about a bit of extra CPU and bandwidth, and assuming that the syntax is the tricky bit of designing a successful, widely deployed, extensible, and highly applicable protocol.

      This is, of course, absolutely the core issue, and saying so highlights what is obviously years of experience in interoperable protocol design.

      We should thank him for his opinion, and drop the decade of work on XMPP, and the twenty further years of experience of the IETF, where text-based protocols also hold sway.

      What do these people know, after all?

      As Adam says, XMPP is just popular because of a sizable core of heavily involved developers. It’d be foolish, in the extreme, to think that the developer mindshare of XMPP is – in any way, shape or form – a product of a clear design, well documented and therefore easily understood.

      No, don’t be fooled – binary protocols are far, far superior, and will win out in the end.

      One only needs to look at the strident successes of X.400 over SMTP, for example. Or FTAM over HTTP.

      Need I go on?

      1. > One only needs to look at the strident successes of X.400 over SMTP, for example. > Or FTAM over HTTP.

        that made me laugh.

      2. Wow, tough crowd eh? Maybe we should take this outside =)? I appreciate Peter Saint-Andre’s insightful and informative comment about the many ways XMPP gets around this issue — every point is right on and is clearly a great decision on your and the rest of the working group’s part. Again, as someone who knows how hard it is to design good working protocols, I have a great deal of respect for the work you’ve done and for what XMPP does in the real world. I also realize what I’m saying is very easy to say with the benefit of hindsight.

        Clearly there’s a progression as you move up the network stack from binary protocols to text-based protocols. Do I think HTTP should be a binary protocol? No, I don’t. Do I think IP, TCP, RTP etc should be text-based? Again, no.

        To me, XMPP is right on that dividing line between when it makes sense to write a binary protocol and when it makes sense to write a text-based protocol. Apparently everyone else on this thread thinks it’s just a slam dunk to put it in text-based land, with my fairly simple arguments holding no merit whatsoever. I find that surprising, but c’est la vie. Also note there are protocols similar to XMPP that went the binary route, most notably Skype and RTMFP — fairly good evidence I’m not the lunatic you seem to want to paint me as being for some unknown reason.

        As far as the new consensus at the IETF that everything be text-based and on the prior useful insight that HTTP is text-based (some joker way up the list, not you Dave), come on — we both know more than that.

        Can’t we just respectfully agree to disagree, or should we rather plunge into the morbid abyss of subtly trying to undermine each other’s technical credibility when we both have better things to do?

        Peter’s point #1 above is perhaps the most intriguing to me. It would be interesting to add a protocol-buffer-like encoding option for XMPP data. That could be interesting.

      3. You see, this is going to be much more dull if I’m forced to be serious.

        No, Peter’s first point was that XML was one of the very few tools available to build something like XMPP at that point in time. The other obvious tool would have been ASN.1, which wasn’t popular at the time, especially in the open-source arena from whence Jabber grew. As it happens, XER/PER comparisons allow us to have some idea of the difference, there, and it’s not that compelling, compared with zlib or EXI.

        Re-encoding the existing XML schema into some serialized form other than XML itself is indeed possible, but it’s tricky to achieve the same flexibility, since XMPP essentially assumes that virtually anything can be extended. As far as any serious experimentation has thus far found out, nothing outperforms simple zlib with sufficient efficiency to be worthwhile. (EXI is an example of something that does help, but is sufficiently complex that nobody can be bothered).

        Skype and RTMFP, being both proprietary protocols, don’t make for good examples here. Skype and Adobe tightly control not only their use, but their applicability, whereas the XMPP community leads the applicability of XMPP by example. In addition, Adobe and Skype really don’t care if their protocol is easy to understand, and – certainly in Skype’s case – they appear to aim for the exact opposite.

        XMPP, on the other hand, has to be very much easy to learn and work with, and experience has shown time and time again that a human-readable syntax really helps to get multiple interoperable implementations. When I said that text-based protocols hold sway at the IETF, I meant that in the RAI and APPs Areas in particular you’d have to work hard to convince people that the benefits of a compact binary protocol outweigh the disadvantages.

        Consider, for example, LDAP. It’s ASN.1/BER, for the good reason that it derives from X.500. There are surprisingly few servers, and most of these are so incestuously related, there’s arguably only really UMich and Quipu, and even being generous, no more than a dozen over all these years. LDAP also never achieved the goal, and has settled into a fairly constrained niche.

        XMPP, on the other hand, has several implementations inside a decade, including by Microsoft, Google, Sun, and now Facebook. It’s had various levels of support from Oracle, Cisco, Yahoo, and AOL – in fact, it’s hard to find any serious company actively avoiding XMPP, now. I suspect a lot of this interest can be boiled down to the protocol being understandable and easily extensible, and both of these in turn owe a lot to there being a human readable syntax.

        So your argument is, basically, that XMPP would be more efficient for specific tasks had it not been written using XML, and instead written using some binary schlock.

        Well, I agree with you, in so far as that argument goes. But I think that protocol efficiency is not the primary goal – interoperability and deployment is, and XMPP has them in spades – so I think it’s self-evident that XMPP is a success, and whilst it might not be the most efficient design possible, that’s outweighed vastly by it being one of the most easily implemented.

        But yeah, do hop on the XSF’s Standards list, and help improve the efficiency – there’s a load of people who’ll welcome the input – and both Peter and I will be amongst them.

        Oh, and FTAM really is better than HTTP. But only because *anything* is better than HTTP, it’s a real crock of a protocol.

      4. Thanks for your thoughtful response, Dave, and for the invitation to pipe up on the list. I’m really only saying all these things because I think rock-solid protocol design now is super important because we could be living with it for awhile.

        Arguments in XMPP’s favor are clearly:

        1) Ease of implementation
        2) Extensibility
        3) Interoperability.

        The interoperability one due to existing implementations is a bit of non-starter if we’re talking about pure protocol design.

        Ease of implementation and extensibility, however, are clearly huge wins, and I really see them as XMPP’s primary advantages. With the advent of things like Protocol Buffers and Thrift, however, this picture changes drastically. If Facebook or Google released messaging and presence protocols built on Thrift or Protocol Buffer message objects, you’d get:

        1) Far smaller messages than XMPP
        2) Forwards and *backwards* compatibility of all messages (which I don’t think XMPP supports?)
        3) Easier implementations than XMPP in every major programming language because all the tricky code is generated.

        The result would be clients and servers that just work better from every angle I can think of save interoperability with clients and servers out there now. They’d be *way* faster. Their code generation in particular is amazing – you’d literally just have to decide the message formats, and you’d have implementations in every major language with one command (save some extra steps with creating the RPC layer with Protocol Buffers).

        Granted, this is a slight diversion from my original argument, but it’s simply astounding what those tools open up in terms of what’s possible to do with minimal effort.

        Anyway, I unfortunately don’t have time to hop on the IETF lists these days, but I think something along those lines would just be super slick and cool (in a really geeky sort of way).

        Best of luck Dave!


  5. From my experience, XMPP is good but not the best design. If we have chance to design an alternative IM protocol, i will choose Google protocol buffer instead of XML.

    The reality is that XMPP is already there, it is not bad and better than other existing ones.

    I have deep experience about XMPP and Google Protocol Buffer.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.