Recently I came across Steve Souder's Velocity 2009 presentation: High Performance Web Sites: 14 Rules for Faster Loading Pages. Steve is an excellent speaker and the author of two important web performance books:

·         High Performance Web Sites

·         Even Faster Web Sites

 

The reason this presentation caught my interest is it focused on 1) why web sites are slow, 2) what to do about it, and 3) the economics of why you should care. Looking first at the economic argument for faster web sites, many companies are obsessed with site performance but few publish data the economic impact of decreasing web site latency.  The earliest data point I recall coming across from a major web site on the price of latency was from the Marissa Mayer 2008 keynote at Google IO:  Rough Notes from Marissa Mayer Keynote. In an example of Google’s use of A/B testing she reported:

[they surveyed users] would you like 10, 20, or 30 results. Users unanimously wanted 30.

·         But 10 did way better in A/B testing (30 was 20% worse) due to lower latency of 10 results

·         30 is about twice the latency of 10

 

Greg Linden had more detail on this from a similar talk Marissa gave at Web2.0: Marissa Mayer at Web 2.0 where he reported:

Marissa ran an experiment where Google increased the number of search results to thirty. Traffic and revenue from Google searchers in the experimental group dropped by 20%.

 

Ouch. Why? Why, when users had asked for this, did they seem to hate it?

 

After a bit of looking, Marissa explained that they found an uncontrolled variable. The page with 10 results took .4 seconds to generate. The page with 30 results took .9 seconds.

 

Half a second delay caused a 20% drop in traffic. Half a second delay killed user satisfaction.

 

Greg reported he found similar results when working at Amazon:

This conclusion may be surprising -- people notice a half second delay? -- but we had a similar experience at Amazon.com. In A/B tests, we tried delaying the page in increments of 100 milliseconds and found that even very small delays would result in substantial and costly drops in revenue.

 

Being fast really matters. As Marissa said in her talk, "Users really respond to speed."

 

The O’Reilly Velocity 2009 Conference organizers managed to convince some of the big players to present data on the cost of web latency. From a blog posting by Souders Velocity and the Bottom Line

·         Eric Schurman (Bing) and Jake Brutlag (Google Search) co-presented results from latency experiments conducted independently on each site. Bing found that a 2 second slowdown changed queries/user by -1.8% and revenue/user by -4.3%. Google Search found that a 400 millisecond delay resulted in a -0.59% change in searches/user. What's more, even after the delay was removed, these users still had -0.21% fewer searches, indicating that a slower user experience affects long term behavior. (video, slides)

·         Dave Artz from AOL presented several performance suggestions. He concluded with statistics that show page views drop off as page load times increase. Users in the top decile of page load times view ~7.5 pages/visit. This drops to ~6 pages/visit in the 3rd decile, and bottoms out at ~5 pages/visit for users with the slowest page load times. (slides)

·         Marissa Mayer shared several performance case studies from Google. One experiment increased the number of search results per page from 10 to 30, with a corresponding increase in page load times from 400 milliseconds to 900 milliseconds. This resulted in a 25% dropoff in first result page searches. Adding the checkout icon (a shopping cart) to search results made the page 2% slower with a corresponding 2% drop in searches/user. (Watch the video to see the clever workaround they found.) Image optimizations in Google Maps made the page 2-3x faster, with significant increase in user interaction with the site. (video, slides)

·         Phil Dixon, from Shopzilla, had the most takeaway statistics about the impact of performance on the bottom line. A year-long performance redesign resulted in a 5 second speed up (from ~7 seconds to ~2 seconds). This resulted in a 25% increase in page views, a 7-12% increase in revenue, and a 50% reduction in hardware. This last point shows the win-win of performance improvements, increasing revenue while driving down operating costs. (video, slides)

 

Souders presentation included many of the cost of latency data points above and included data from the Alexa Top 10 list to show that the bulk of web page latency is actually front end time rather than server latency:

  

Steve’s 14 rules from his book High Performance Web Sites:

1.       Make fewer HTTP requests

2.       Use a CDN

3.       Add an Expires header

4.       Gzip components

5.       Put stylesheets at the top

6.       Put scripts at the bottom

7.       Avoid CSS expressions

8.       Make JS and CSS external

9.       Reduce DNS lookups

10.   Minify JS

11.   Avoid redirects

12.   Remove duplicate scripts

13.   Configure ETags

14.   Make AJAX cacheable

 

I’ve always believed that speed was an undervalued and under-discussed asset on the web.  Google appears to be one of the early high-traffic sites to focus on low latency as a feature but, until recently, the big players haven’t talked much about the impact of latency. The data from Steve’s talk and his blog entry above is wonderful in that it underlines why low latency really is a feature rather than the result of less features. The rest of his presentation goes into detail into how to achieve low latency web pages. It’s a great talk.

 

                                                                --jrh

 

James Hamilton

e: jrh@mvdriona.com

w: www.mvdirona.com

b: blog.mvdirona.com / perspectives.mvdirona.com

Saturday, 31 October 2009 07:39:37 (Pacific Standard Time, UTC-08:00)      Comments [6] - Trackback
Services

Saturday, 31 October 2009 09:53:15 (Pacific Standard Time, UTC-08:00)

It feels like we've come full circle. To accommodate modem economics, early Yahoo pages exhibited html post-processing and as much stripping of then-valid markup as browsers would bear.

web.archive.org/web/19991116151216/www4.yahoo.com/

It's worth mentioning Souders comes out of Yahoo's Performance group, his only the most-known of good work there.

Jeremy Wohl

Saturday, 31 October 2009 12:14:24 (Pacific Standard Time, UTC-08:00)

Yes, you are right. Steve Souders became very well known prior to joining Google a couple of years back.

--jrh
jrh@mvdirona.com

James Hamilton

Monday, 02 November 2009 15:09:00 (Pacific Standard Time, UTC-08:00)

Velocity was good this year, Souders is Mr. Web Performance - For me, the Shopzilla presentation was a highlight - here's a video of it - Shopzilla's Site Redo - You Get What You Measure blip.tv/file/2290648 . Also a presentation by a guy from Aptimize regarding latency and the website accelerator WAX was amzing, sorry don't know if that was recorded.

Tuesday, 03 November 2009 06:25:11 (Pacific Standard Time, UTC-08:00)

Thanks for the pointer to the Shopzilla video JakeF.

--jrh

James Hamilton

Thursday, 12 November 2009 13:14:31 (Pacific Standard Time, UTC-08:00)

Hi

Thanks for sharing that data, it is good to see some $$$ amount associated with improvements in response times.

- Andrew

Andrew

Thursday, 12 November 2009 16:27:43 (Pacific Standard Time, UTC-08:00)

Yes, I've been collecting little bits of evidence in support of the perspective that low latency was a feature in and of itself rather than what was traded off to implement features.

Velocity this year was a gold mine of great data. It's good to see more hard data published.

--jrh
jrh@mvdirona.com

James Hamilton