Comet: Low Latency Data for the Browser
An old web technology is slowly being resurrected from the depths of history. Browser features that have gone untouched for years are once again being employed to bring better responsiveness to UIs. Servers are learning to cope with a new way of doing things. And I’m not talking about Ajax.
New services like Jot Live and Meebo are built with a style of data transmission that is neither traditional nor Ajax. Their brand of low-latency data transfer to the browser is unique, and it is becoming ever-more common. Lacking a better term, I’ve taken to calling this style of event-driven, server-push data streaming “Comet”. It doesn’t stand for anything, and I’m not sure that it should. There is much confusion about how these techniques work, and so using pre-existing definitions and names is as likely to get as much wrong as it would get right.
Defining Comet
For a new term to be useful, at a minimum we need some examples of the technology, a list of the problems being solved, and properties which distinguish it from other techniques. As with Ajax, these aren’t hard to find. A short list of example applications includes:
- GMail’s GTalk integration
- Jot Live
- Renkoo
- cgi:irc
- Meebo
So what makes these apps special? What makes them different from other things that might at first glance appear similar? Fundamentally, they all use long-lived HTTP connections to reduce the latency with which messages are passed to the server. In essence, they do not poll the server occasionally. Instead the server has an open line of communication with which it can push data to the client.
From the perspective of network activity, we can modify JJG’s original Ajax diagram to illustrate how Comet differs:
As is illustrated above, Comet applications can deliver data to the client at any time, not only in response to user input. The data is delivered over a single, previously-opened connection. This approach reduces the latency for data delivery significantly.
The architecture relies on a view of data which is event driven on both sides of the HTTP connection. Engineers familiar with SOA or message oriented middleware will find this diagram to be amazingly familiar. The only substantive change is that the endpoint is the browser.
While Comet is similar to Ajax in that it’s asynchronous, applications that implement the Comet style can communicate state changes with almost negligible latency. This makes it suitable for many types of monitoring and multi-user collaboration applications which would otherwise be difficult or impossible to handle in a browser without plugins.
Why Is Comet Better For Users?
Regular Ajax improves the responsiveness of a UI for a single user, but at the cost of allowing the context to go “stale” for long-lived pages. Changes to data from others users is lost until a user refreshes the whole page. An application can alternately return to the “bad old days” and maintain some sort of state mechanism by which it tells client about changes since the last time they’ve communicated. The user has to either wait until they preform some action which would kick off a request to see the updated state from other users (which might impact the action they wanted to preform!) or request changes from the server at some interval (called “polling”). Since the web is inherently multi-user, it’s pretty obvious that regular Ajax imposes usability and transparency hurdles for users. Applications that employ the Comet technique can avoid this problem by pushing updates to all clients as they happen. UI state does not go out of sync and everyone using an application can easily understand what their changes will mean for other users. Ajax improves single-user responsiveness. Comet improves application responsiveness for collaborative, multi-user applications and does it without the performance headaches associated with intermittent polling.
But Does It Scale?
New server software is often required to make applications built using Comet scale, but the patterns for event-driven IO on the server side are becoming better distributed. Even Apache will provide a Comet-ready worker module in the upcoming 2.2 release. Until then, tools like Twisted, POE, Nevow, mod_pubsub, and other higher-level event-driven IO abstractions are making Comet available to developers on the bleeding edge. Modern OSes almost all now support some sort of kernel-level event-driven IO system as well. I’ve even heard that Java’s NIO packages will start to take advantage of them in a forthcoming release. These tools are quietly making the event-driven future a reality. This stuff will scale, and most of the tools are in place already.
I’ll be giving a more on this topic at ETech and describing the various techniques that Comet applications can employ to push data from the server to the client. As always, I’ll post the slides here as well.
The future of the read-write web is multi-user. There is life after Ajax.
Endnotes
First, a word on terminology and its importance. “Ajax” was coined to describe background request/response data transfer. Many of us had worked on solutions to do exactly this, but it wasn’t until a simple name and accompanying description were provided that it was possible for people not directly building applications to describe what it was they liked about it. Common terminology acts not only as a shortcut in discussions between technical folks, but also as a bridge for those who may not be able to give a technical rundown of exactly how it works.
As with Ajax, those of us who build technology are now faced with another communication challenge. We have a hard problem for which solutions are available (and have been for some time) but no way to communicate about them. Terminology is again the missing link. Today, keeping an HTTP connection open for doing low-latency data transfer to the browser has no digestible name. When I describe a cool new hack, there’s nothing to associate it with. When people say “how the hell did they do that?”, we don’t have a compact answer. Therefore, in the spirit of improved communication (and not technology invention), I’m proposing a new name for this stuff.
Next, for those who are network-level programmers or are familiar with sockets and/or basic TCP/IP programming, you will probably scoff at the concept of web applications finally getting this kind of datagram packet support. Fair enough. It is however interesting to note that while more responsive UIs have been available on a variety of platforms to date, the Web has “won” the broad majority of market share for most classes of applications in which the browser provides enough native (non-plugin) support to make the performance and/or UI feasible. Comet may be a new name for an old set of concepts wrapped in some pretty grotty hacks, but that in no way diminishes the market impact it will have (and is already having).
Lastly, as current Dojo users might expect, Dojo already supports Comet via dojo.io.bind(). More than a year ago we designed the API with Comet in mind. In the next couple of weeks I’ll be showing how bind‘s pluggable transport layer can be combined with Dojo’s event topic mechanism to provide message delivery on top of a message bus.
139 Comments
I wish we had a public version of AjaxWar up. Wouldn’t it qualify as Comet?
This is great stuff. I work on DOD weather applications and have had to implement similar “hacks” to support this type of transfer. I believe we will be moving our stuff to dojo in the near future
Alex:
Will you be presenting on Comet at the Ajax Experience?
Isn’t this just HTTP 1.0 style keep alive? I believe KnowNow implements this with HTTP 1.0… anyway… now we have something to call this. PeopleSoft apps use this style of communication for their Help Desk online chat apps (they OEM’d KnowNow’s pub/sub tools back in 2001). Like XMLHTTPRequest, it’s nothing new… but now we have a name for it… cool name btw.
Rich
Jeff: yes, I think AjaxWar qualifies
Rob: If you like!
Rich: yes, sort of. The difference is that you don’t initiate a new HTTP transaction for every datagram. Further, the client doesn’t initiate the connections. In the HTTP 1.1 model, a TCP connection is indeed held open for multiple requests, but it is always the client requesting discrete blocks of content and then rendering them once the transaction is complete. Comet breaks these assumptions.
So how well will a server cope with 1000′s of open sockets? Obviously, the problem of the server having a thread allocated (for threaded servers) will have to be fixed, but assuming they are, and we learn the new models of how to deal with this, I’m wondering if servers will really be able to handle the sheer number of open sockets.
There’s definitely something to be said for stateless …
Pat: If your server is using a threaded network responder layer, you’re pretty effed. OSes will handle that many open (10K+) fd’s without any problem whatsoever, however the deamons that read/write from them often run into scalability issues. Select and poll are O(n) in these cases, and we really want something closer to O(1). Linux’s epoll and FreeBSD’s kqueue give this to us at a kernel IO level. Your web daemon has to take advantage of them, however.
This is where tools like Apache’s mpm\_event, Twisted Python, and POE come into play. They are event driven web daemons. That is to say that they don’t allocate an OS thread or process per connection. Instead, they deliver data asynchronously to waiting listeners on an event driven basis. These are the kinds of tools that let your server handle this kind of load without breaking a sweat. If, OTOH, you go with something like a traditional Apache+mod_whatever infrastructure, you’re just not going to scale.
Consider that your traditional web server is designed to take a request, satisfy it as fast as possible, and close the connection and you can easily see that they just aren’t optimized for this kind of workload. I expect that all of our tools will lurchingly come up to speed for use with things like Comet, but the transition isn’t going to be pretty for folks who don’t grok the problems (and the potential solutions).
Regards
time for yaws?
one day the crayon crowd is going to have to start thinking about profiling and performance. be it insane server loads (for comet) or insane load on the client (massive DOMs, canvas, SVG, etc), users may start balking at some of this frosting.
and once again i will offer the offtopic comment i always offer when i see “meebo” – you are insane if you hand over your password to a third party, and frankly the manin IM services should block them. meebo is not gaim or trillian – they connect directly with the authentication servers from the IM provider.
this page also provides a nice write-up of comet (before it was called comet)
Yet another great article (YAGA) Alex.
A quick question about “Even Apache will provide a Comet-ready worker module in the upcoming 2.2 release”. To what module are you referring to as Apache 2.2.0 has been out for some time now.
I’d love to see a small demo of using Comet with Dojo if anyone has one.
As for handling large numbers of open connections I’d look into [Open]Solaris 10. It introduces a highly scalable and enhanced networking stack, which lowers overhead by reducing the number of instructions required to process packets. This efficiency also increases scalability, allowing more connections and enabling server network throughput to grow linearly with the number of CPUs and NICs.
grumpY!: is Google a member of the “crayon crowd”?
Just because your stock web server can’t do it today, that doesn’t mean it can’t be made to scale. While we weren’t looking the OSes picked up the slack and things like libevent got written. The only really surprising thing is that it didn’t happen back when dialup users (whose longer-lived connections look similarly like Comet) ruled the roost.
The biggest scalability issue is going to be for people running poorly written proxies. Regardless, I don’t think that reflects on the engineering ability (CPS resurrected!) of those building the servers to host Comet apps. The proxies will catch up.
You should give Twisted a long, hard look. It barely breaks a sweat on most of these workloads.
Regards
That *is* a very good writeup.
Steve: last I checked the event\_mpm wasn’t marked stable yet? I hesitate recommending people use something like that when things like Twisted exist and are stable.
alex- this is why i mentioned yaws. erlang has been providing insane threading support for some time now, the yaws graph is mind blowing. soon erlang will have smp. with multicore being the order of the day, erlang will either rise on its own or influence others. i would argue that yaws could handle this model gracefully.
is google in the “crayon crowd”? google gets a free pass because they have an unlimited hardware budget. that said, i suspect they are in fact doing tuning and profiling. the others…who can say? meebo’s traffic is a rounding error.
grumpY!: I wasn’t aware of yaws. I’ll have to check it out. That also reminded me of the IO language that the author of LivePage was raving about recently.
/me revises ETech slides
alex – this is the yaws vs apache graph i spoke of.
once again, people doing serious http streaming should look at yaws
I’d love to see how this stacks up against lighttpd, Twisted Web and the event\_mpm.
>> I’d love to see how this stacks up against lighttpd, Twisted Web
ericsson runs telco switches on erlang, as do some other telcos, so my guess is that erlang would demolish these tools by a considerable margin, which is why yaws can stay alive long after apache has fallen over due to too many connections.
erlang runs as one process and manages its own threads. this is why it can offer “better than OS” thread performance.
FWIW, Twisted does the same thing. It uses CPS-style programming to keep everything in a single process.
I would still like to see the benchmarks = )
The expensive web farm load balancers don’t have much to do anymore.
You must be freaking kidding. (I wish I could use a swear word there.)
But this technology has been available for a long time and all you are trying to do is attach your name to it, by labeling it. Trust me, there are plenty of people already creating applications by utilizing the long lived http connections. Keep your stupid label “Comet” to yourself, all we need is more tards hopping on this bandwagon, like the Ajax one.
Mike: if you had read the post in it’s entirety, you’ll see that I both understand the origins of the technique and acknowledge them. I personally worked on a re-write of the mod_pubsub client 2+ years ago. It was old technology then.
What is different today is that it’s being widely deployed.
As for my personal affiliation with it, I would only like to see its use become more common. I’m quite happy to continue hacking on Open Source DHTML in its various forms. I’m not trying to sell you or anyone else anything.
The only shot the web has at remaining open is for it to deliver better user experiences. If terminology allows us to do that before it becomes co-opted by any number of the large companies currently salivating at the prospect, so be it.
I’ll be your whipping boy if it’s better for the web.
To go from client-poll to server-push it seems everybody has forgotten about multipart/x-mixed-replace. It is sitting there ready for use since years – no HTTP overhead, minimum latency. Php example on: php.net/flush look for comment from 20-Nov-2005 04:06.
AFAIK it is supported together with xmlhttprequest by firefox (but not IE6).
The problem will be the inter process message passing on the server though, as in today’s scripting web (PHP, ASP, etc.) there is a copy of the program running for every client connection.
Second problem will be handling several thousand parallel connections at once.
Nevertheless, thanks for the cool article.
Just a note to mention that Nevow Athena implements this kind of bi-directional client-server communication link. As far as coping with “1000s of open sockets”, the problem isn’t any more severe than any other kind of protocol where connections are persistent and long-lived; true, a lot of web tools today may not be geared up to handle this mode of operation, but this certainly wouldn’t be the first time anyone has had to deal with it.
We included “comet” functionality into Millstone 3.0 UI component library released back in 2002. Basically you could you do any changes to components on server and they are automatically pushed to client through already open pipe. Unfortunately making this communication robust across all browsers is challenging. For an online demo and source-code, see “Game of Go” -demo on librarys online demo-page: examples.millstone.org/
Marcus: there is actually an analagous XML envelope available in IE. It’s not multipart (which I think would be preferred), but it beats a kick in the teeth. It’s also possible to poll responseText before the connection is closed. Regardless, I’m still more a fan of the hidden br technique since it allows you to set document.domain and avoid gobbling one of your precious connections when served from a subdomain.
I wish I had this about 6 months ago! I had to develop my own push based system based on pushlets to implement a real-time auction system.
Good to know this type of functionality is moving forward
Comet is a stupid name, that is the name of bathroom cleaner. Don’t give up your day job guy. I will use your comet to clean my toilet and serve my pages of the web. Good technology, stupid name. Why not call it mullet or bandaid© or mr. clean© ??? I realize that AJAX is name of cleaner too, but at least stands for something, but comet? It is lame and annoying. I see no humor or irony in this. Excuse my English, I from USA.
haha…I missed the obvious cleanser pun until I read bronze_web’s comment. The whole time I was thinking of a meteor shower of data coming from the server. I like the name, though it does seem to not do justice to the bi-directional nature of the communication link (i.e., if you’re thinking of falling meteorites). Let’s see if it sticks.
No this isnt new, but I don’t think anyone was really claiming it was. But just like Ajax it is becoming more popular and can only benefit from things like this.
My company has been doing this for over 8 years in various ways. Starting off with Java applets/LiveConnect, and moving on to pure JavaScript implementations using both multipart replace and br script tag methods.
It is certainly possible to acheive the necessary performance on a server, numbers of open sockets are not really a problem if implemented well, eg up to 30000 concurrent users. The problems are more related to the amount of data sent to the clients. Low message rates are fine, but to increase the messages rate you have to be more intelligent with the protocol and other techniques. With decent message rates overall bandwidth can be high.
I have read the JS source code for Jot Live. I find the nature of Comet is quite similar with AJAX. The only difference between them is that the Comet will start a new XMLHttp request immediately after the previous one finished.
Would it be a burden for the server? Due to the frequently requests.
Jetty 6′s continuations would probably work well with this technique.
orbitz-erlang.blogspot.com/2005/09/twisted-matrix.html
some more intelligent notes on erlang vs twisted
i wrote a very small chat application with nevow’s athena. i use vhosts to overcome the 2- connection-limit per host (at least it seems to work). chat.sifu.co.at
Just call it a “Hanging-GET” because that’s what it really is.
“Comet” (gag) is just a new name for Pushlets.
www.pushlets.com
Granted, the new name is intended for marketing purposes, but let’s not forget it’s the exact same idea with a different name. Both Pushlets and Comet both depend on a Hanging-GET (which I think is a better, more technology neutral term anyway.)
Don’t forget also that this can’t really be used as a generic eventing mechanism because it doesn’t support gauranteed delivery. Most applications don’t require it but there are a lot that do.
I’d like to see an implementation of HTTPLR
www.dehora.net/doc/httplr/draft-httplr-01.html
which woudl provide gauranteed delivery. It would require explicity acknowlegement of receipt, but with HTTP keep-alive, you’d end up talking over the same sockets anyway. So you still have the same threading issues to deal with but it’s a more robust solution for a general client.
HTTPLR looks interesting, but that doesnt look like it could support Comet/Hanging-GET style push communication since all the gauranteed messaging is done on standard HTTP request/response headers. In comet you want to send multiple messages from server to client within the same response. Is HTTPLR going to address this?
I am really looking forward to your follow-ups on how dojo supports comet…I’m ready to try that already!
Please take a look at www.lightstreamer.com too.
Several online demos are available.
Lightstreamer is a commercial poduct that has five years of maturity and very soon a free edition will be released.
It can be integrated with any AJAX framework to implement true “push/streaming” capabilities, for updating a page with real-time data without relaying on any form of polling.
Lightstreamer Server is highly scalable, due to its NIO-based staged event driven architecture.
Any feedback will be appreciated.
ICEfaces is a commercial product that deploys “comet” applications that are developed in pure Java (no JavaScript) using standard JSF components.
There is a free Community Edition available as well as a Professional Edition (commercial license). One of the advantages of the commercial version is that it provides a highly-scalable solution to the thread-consumption problem identified above related to holding connections open on the server.
Check it out for an example of a complete comet application development and deployment solution.
Online Demos: www.icesoft.com/products/demos_icefaces.html
ICEfaces information:
www.icesoft.com/products/icefaces.html
Regards,
Ken
TelePort is an Open Source product that supports Comet, JSON, SOAP, REST etc. etc.
(For the veterans amongst you guys: it is an evolution of the 2001 vcXMLRPC library)
It is released under the GPL and available on Sourceforge:
sourceforge.net/projects/teleport/
For more information, wander over to
www.javeline.org/modules/products/
I think ‘Comet’ is as good a name as any other. Using Ajax on my kitchen floor did not make me confuse it with AJAX which I use in my work.
I do see why ppl immediately brought up performance issues with open connections. I’m sure this will be solved soon. AJAX spread so much faster since implementing only involved some php class and a couple lines of jacascript. But with ‘Comet’ it seems I need some apache module, or a similar technique outside of the reach of the ordinary web-developer.
Right now I would see the use of it in the su