PHP jQuery AJAX Javascript Long Polling

04 May

by: Matt in Development, JavaScript, PHP

tags: ajax, JavaScript, lighttpd, long polling, nginx, php

Background

"Long polling" is the name used to describe a technique which:

• An AJAX request is made (utilizing a javascript framework such as jQuery)
• The server waits for the data requested to be available, loops, and sleeps (your server-side PHP script)
• This loop repeats after data is returned to the client and processed (usually in your AJAX request's onComplete callback function)

This essentially simulates a continuous real-time stream from the client to the server. It can be more efficient than a regular polling technique because of the reduction in HTTP requests. You're not asking over and over and over again for new data - you ask once and wait for an answer. In most cases this reduces the latency in which data becomes available to your application.

There are a variety of use cases in which this technique can be handy. At the top of the list are real-time web-based chat applications. Each client executes a long polling loop for chat and user events (sign on/sign off/new message). Meebo is perhaps the greatest example of this.

It's important to note some of the server side technical limitations of long polling. Because connections remain open for considerably longer time than a typical HTTP request/response cycle you want your web server to be able to handle a large number of simultaneous connections. Apache isn't the best candidate for this type of situation. nginx and lighttpd are two lightweight web servers built from the ground up to handle a high volume of simultaneous connections. Both support the FastCGI interface and as such can be configured to support PHP. Again, Meebo uses lighttpd.

For similar reasons - it's also a good idea to choose a different sub-domain to handle long polling traffic. Because of client side browser limitations you don't want long polling connections interfering with regular HTTP traffic delivering page and media resources for your application.

Implementation

jQuery makes implementation a breeze.

var lpOnComplete = function(response) {
	alert(response);
	// do more processing
	lpStart();
};

var lpStart = function() {
	$.post('/path/to/script', {}, lpOnComplete, 'json');
};

$(document).ready(lpStart);

Straightforward. When the document is ready the loop begins. Each iteration the returned data is processed and the loop is restarted.

On the server side - just like we discussed earlier:

$time = time();
while((time() - $time) < 30) {
	// query memcache, database, etc. for new data
	$data = $datasource->getLatest();

	// if we have new data return it
	if(!empty($data)) {
		echo json_encode($data);
		break;
	}

	usleep(25000);
}

Actually, a couple points of interest here. We don't actually loop infinitely server side. You may have noticed the logic for the while loop - if we've executed for more than 30 seconds we discontinue the loop and return nothing. This nearly eliminates the possibility of substantial memory leaks. Also, if we didn't put a cap on execution time we would need to print a "space" character and flush output buffers every iteration of the loop to keep PHP abreast to the status of this process/connection. Without output being sent PHP cannot determine if the connection was lost via connection_status() or connection_aborted(). As a result this could lead to a situation where there are an increasing number of "ghost" processes eating up server resources. Not good!

That pretty much sums it up! Not that difficult, right?

As always, questions/comments are welcome, hope this helps!

Related posts:

1. Real-time “AJAX” JavaScript Progress Bar
2. PHP Dynamic JavaScript SCRIPT Insertion for Embedding
3. PHP fast, large (megabyte), data transfer between sessions
4. PHP Forking to Concurrency with pcntl_fork()
5. jQuery 1.4 Released

• rowanhenderson.com Rowan

  Hello. This is the best source of info I have found on this technique using jQuery, but I have a quick question. what would be a good example to use for $data = $datasource->getLatest();

  currently when i make getLast() output anything, it will just make the loop go extremely fast and output all the data, even if I make it statically output the number 4, it will still loop it, even though the data is the same… does this make sense?

• Matt

  @Rowan

  The purpose of that line of code would be where you would check a datasource (memcache, mysql db, etc.) for new data. In a chat application your app will receive events from chat clients (new msgs, status updates, etc.). You would need to do some logic in this method to check to make sure you aren’t sending the same data over and over again.

  You only want to output something (and break execution of the “infinite” loop) if theres something that NEEDS to be output. Notice the “break;” within the if. Essentially its only going to echo output and break the loop if the data returned from the getLatest() call isn’t empty (as in, “we have new events”).

  I think the reason why your loop is going really fast is exactly because you’re returning a constant like the number 4, which ISNT empty. The loop echo’s, break’s and jQuery restarts it.

  Does this help?

• John

  Looks promising, is it possible to expand this out a bit? Perhaps offer some demos/examples? I'm trying to learn long polling, and I reckon this would help me crack it.

• matticakes

  John,

  Are you having trouble tailoring the example code I provided to your domain? What are you trying to accomplish?

• adrien

  Interesting but what happen if you implement this technique on a page that have other AJAX features ?

  ajax requests will be queued until no response received from lpStart :x

• adrien

  mmmh that's maybe why sub-domain is a good idea, I'll try =)

• Way

  I'm working for a couple of weeks with that kind of ajax techniques. At the moment I'm standing before a big problem. Let me explain:
  As in this implementation above I use an while-loop on server side. Before i start this while, I start the session and check, if the current user is logged in.
  So here is my problem. I find out that the session is the reason for my problem. Cause if I break the client request via hitting F5 to refresh the page and my script loads again, it have to wait until the server leaves the while-loop.
  For example, i poll the server and my while looks exactly like that above and runs 30 seconds.
  If I refresh my page and make a new request to server after 5 seconds, my client has to wait for the rest of 25 seconds until the server recognize the new request and starts the while again.

  This is quite not that what I want. Have anyone an idea or solution?
  I've tried things like “ignore_user_abort(false)” but that's not the answer of my problem.

  btw. I'm from Germany so sorry for my very own version of English ;)

• matticakes

  Way,

  I believe this has to do with PHP Sessions using the default “files” save_handler. In order to prevent race conditions with data inside the $_SESSION variable PHP locks the file its storing the data in. A second request that tries to initiate a session tries to access this same file and waits for it to become available.

  If you use a custom session save_handler (like memcache or your database) and implement variable level locking (instead of session level locking) you can circumvent this problem.

  Perhaps this is a good blog post, I'll write something up.

  Thanks,

  -Matt

• Way

  Thank you very much Matt,

  I've suspected that. Actually I try to work with comet and an br-technique. But i think there is that problem too. So I will definitely need a custom session handler.

• Dave

  Thanks for writing this, it's pretty much what I'll be working on over the next few weeks. One question I have though is we're running Apache 2.2.11 and PHP 5.2.9 on a cPanel box. I don't want to switch to another web server if I can help it, though cPanel does allow you to install FastCGI. Initially HTTP requests to/from the server will be low, but we have a fast enough box to support the load. Would FastCGI help us out in this instance?

  Thanks in advance,

  Dave

• matticakes

  Dave,

  No, using FastCGI as an interface for PHP and Apache wouldn't help. It's Apache itself thats the problem. It isn't a web server meant for a high-volume of simultaneously open connections.

  It isn't that difficult to get up and running with lighttpd on a cPanel box.

  -Matt

• Dave

  What about LiteSpeed with FastCGI support? Any known issues with that or does it fall under the same category as Lighttpd and nginx?

  Thanks,

  Dave

• matticakes

  Dave,

  I think you're thinking about this wrong. You don't need FastCGI in order to scale an app that uses this technique. FastCGI just happens to be a common interface of servers that are able to scale with an app using this technique.

  Lighttpd would be my suggested choice given that it's easy to get PHP up and running on it and is fairly easy to install.

  -Matt

• Dave

  OK, thanks Matt, much appreciated.

• devezine.com/php-jquery-ajax-javascript-long-polling-perplexed-labs/ PHP jQuery AJAX Javascript Long Polling | Perplexed Labs

  [...] the original here: PHP jQuery AJAX Javascript Long Polling | Perplexed Labs Share and [...]

• Az

  Try session_write_close(); I wasted 2 weeks of my life to get rid of it, But session_write_close() helped me. Problem was exactly the same.

• d1Mm

  > sleep(25000);
  it's about 7 hours ;)
  may be usleep(25000) ?

• imwoodie

  Hi, I'm using jquery-1.3.2.. I try to implement your code and is running well in FF, safari, and opera. but it doesn't work in IE.. would you like to give me a clue.. thank you

• blog.perplexedlabs.com matticakes

  have you tried enabling script debugging to see what line of code is causing the problem?

• imwoodie

  thanks for reply matt.. appreciate.

  well there's no error code in I.E, I'm not using jason but xml as a data..
  here are code that I try to implement :

  
var lpOnComplete = function(response) {
$(response).find('id').each(function(){
var id = $(this);
bidid = id.attr("id");
alert('bidid');
// do more processing
});

  lpStart();
  };

  var lpStart = function() {
  //$.ajaxSetup({ cache: false });
  $.get('proc.php', { }, lpOnComplete, 'XML');
  };

  $(document).ready(lpStart);


  This is proc.php, I'm using memcache

  
<?php
$time = time();
while((time() - $time) < 30) {

  // query memcache, database, etc. for new data
  $memcache_obj = memcache_connect('127.0.0.1', 11211);
  $data = "<?xml version='1.0' encoding='iso-8859-1'?>
  <data>" . memcache_get($memcache_obj, 'bid') . "</data>";

  // data is always new
if(!empty($data)) {
Header("Content-Type: text/xml");
echo $data;
break;
}
usleep(2500);
flush();
}
?>


  It doesn't work in I.E

  Thanks again matt

• blog.perplexedlabs.com matticakes

  A couple things.

  You should move your memcache connection line outside the while loop – you don't want to reconnect every iteration.

  $data, the way you have it written, can never be empty so the loop will always only run one iteration.

  If there are no script errors being reported, is an empty alert box being displayed?

• imwoodie

  thanks for correction in memcache matt..

  sorry, the code above was alert(bidid) not alert('bidid');
  the alert box is displayed well with returned data from xml
  it works in all browser except in I.E all version
  in I.E the alert box is not displayed.. that's the problem

  is there any special code for I.E?
  how to make it works in all brower matt with no exceptiom,
  I'm getting stuck with it.

  Thanks

• Marko Jovanovic

  Thank you… Thank you…Hey did i say THANK YOU!
  You probably saved 2 weeks of my life :)

• davidwoodfield.wordpress.com/2009/11/24/resources-for-php-webmail/ Resources for PHP Webmail « David Woodfield

  [...] blog.perplexedlabs.com/2009/05/04/php-jquery-ajax-javascript-long-polling/ [...]

• www.facebook.com/people/TJ-Mapes/16913244 TJ Mapes

  Wow great post, does this FB Work?

• www.facebook.com/people/TJ-Mapes/16913244 TJ Mapes

  Wow great post, does this FB Work?

• mtthwkfmn

  Hello,

  Awesome post on jQuery and PHP implementation!

  I like the 30 second handler implementation…really simple and good idea to handle it there…

• twitter.com/Mirgorodo Mirgorod Vadym

  It is very nice. I'll try this tommorow. Thanks to author :)

• Vdecaux83

  Nice article, but can we make an other ajax-call ? While the ajax-long-polling is running ?

  I don’t think so, so we block any ajax actions ? Or I am wrong, I have just tried, and I’ve got this error.
  Thanks !

• NoICE

  You need to add “async:true” to config options for the main ajax request

• Kal

  Great stuff!

  I have successfully implemented this solution. However if I have multiple tabs open the notifications to duplicated. Is there any way to prevent this?

• Kal

  Hi, I have successfully implemented the above solution however if i have multiple tabs open then each tab will have a polling process. Is there a way to get around this as really I only want to poll on one tab.

  Thanks

• Jason

  Is possible to refactor this to use shorter code and pure JavaScript. Not sure which is best to accomplish simple long polling JavaScript example, but I think I found it on that blog. Thanks!

 Subscribe in a reader

ajax apache asynchronous backup c++ centos cron CSS curl deployment Development Django firefox framework getcwd java JavaScript jquery languages libwkhtmltox logs long polling memcache mod_wsgi mysql m