When Is Your Web Site Fast Enough?

Web usability guidelines aim to help you keep and serve your customers, rather than driving them to your competitors’ sites out of frustration. Usability is important for every Web site, but becomes absolutely vital when a large portion of your revenue comes from Web sales. Over the years, I’ve read many articles and checklists about Web site usability, and they always mention the importance of site responsiveness.

How does responsiveness stack up against other aspects of site usability? There are many ways to slice the usability pie, and each author seems to offer a different taxonomy; here is mine. To satisfy customers, a Web site must fulfill four distinct needs: availability, responsiveness, clarity, and utility. This list presents the four essential qualities in order of their significance, and applies to all first-time visitors.

A site that’s unreachable, for any reason, is useless. Having reached the site, pages that download slowly are likely to drive customers to try an alternate site. But if the site is sufficiently responsive, other design qualities come into play. Using the site must be simple and natural — it must be easy to learn, predictable, and consistent. Last comes utility — does the site actually deliver the information or service the customer was looking for in the first place?

Once a customer has established a reason to stay on a site, their priorities may change. But only a prior knowledge of some unique utility not obtainable from other sites can overcome frustrating slowness or poorly designed navigation features. If you accept this analysis, then the importance of responsiveness is clear.

Service Level Management (SLM)

Knowing that page download times are important is just the beginning. Building and maintaining first class e-business applications demands a systematic commitment to delivering levels of quality that can be measured and managed. Companies must address many inter-related issues, including:

• What level of performance do our customers really expect?
• How can we match, even stay ahead of, the competition?
• How will we prepare for our next big sales event (or season)?
• How will we measure site and application responsiveness?
• How will we know when our customers experience a drop in service levels?
• How do we diagnose and fix problems quickly?
• How will we monitor, quantify, and report on our success?

The management and technical activities required to tackle these issues are collectively called Service Level Management, or SLM. To implement SLM successfully, many people with diverse skills and responsibilities must contribute, because SLM touches every aspect of the application lifecycle — site design, database design, application programming and testing, systems management, and networking.

This is a broad topic; if you would like to read a book about it, I recommend Practical Service Level Management: Delivering High-Quality Web-Based Services by John McConnell and Eric Siegel (Cisco Press, 2004).

Although SLM actually involves managing both site availability and site responsiveness, I will focus only on the latter. The costs of site downtime are relatively easy to quantify: when customers can’t reach your site, you are losing business. And even if you can’t assign a precise dollar cost to every outage, at least everyone understands why the ultimate goal is 100 percent uptime, and why 90 percent is likely to be ten times more damaging to your business than 99 percent.

A cost/benefit analysis of responsiveness is a lot trickier. Faster may be better, but how fast do you really need to be? And even if you have set a target, what is the cost of missing it? How much should you invest in speeding up your site? These are the harder questions facing e-commerce executives today. Here’s a framework for thinking about them.

How Long Will People Wait?

Research shows that Web users’ tolerance for delays depends on several factors, including their expectations, site feedback, the complexity of a task, its importance, and the relevance (utility) of the information being provided by the site. And their perception of a site’s quality and credibility diminishes as its download times increase.

As long ago as 1968, when all computers were mainframes, Robert B. Miller’s classic paper on “Response Time in Man-Computer Conversational Transactions” described three threshold levels of human attention. A response time of one tenth of a second is viewed as instantaneous, a response within 1 second is fast enough for users to feel they are interacting freely with the information, and response times must stay below 10 seconds to keep the user’s attention focused on the dialog. Miller also concluded that a consistent 2 second response would be ideal.

In 1997, Peter Bickford’s landmark paper, “Worth the Wait?,” reported research in which half the users abandoned Web pages after a wait of 8.5 seconds. Bickford’s paper was quoted whenever Web site performance was discussed, and the “8-second rule” soon took on a life of its own as a universal rule of Web site design.

Many researchers have investigated the subject of Web responsiveness since Bickford, but no new universal rules have emerged. This is no surprise, because Miller’s findings were a direct result of how people’s brains operate, and they applied to any human interactions with machines. Changing the machine from a mainframe terminal to a Web site has not changed people’s brains.

So how should we regard Miller’s thresholds? On the Web today, the home page download times of the top 10 or so sites on the Keynote Business 40 index do actually achieve Miller’s 1-second threshold (for users with a high-speed connection). More pages achieve Miller’s 2-second guideline, but it’s still safe to say that the vast majority does not.

While Miller’s findings identified some important (and invariant) behavioral thresholds, it is evident that people’s satisfaction with their experience of an individual Web site is determined by more than those thresholds alone. We do not judge every interaction against a 2-second threshold; credit card validation is a good example. Why not?

The key additional ingredient is people’s prior experience of the Web environment as a whole. Jakob Nielsen’s “Law of the Web User Experience” points out that people spend most of their time on other Web sites. Their prior experiences set their expectations and provide the context in which they can judge subsequent online interactions. Putting a new twist on an old acronym, we might say that as a Web user, what you suppose is what you get (WYSIWIG).

Delight, Satisfy or Frustrate

What does this mean for an e-commerce company embarking on an SLM program? How should you set response time objectives?

First, forget the 8-second rule! What really matters are the service levels your customers expect. You can delight, satisfy, or frustrate them by delivering levels of responsiveness that they perceive to be fast, reasonable, or slow respectively.

Those perceptions are going to be set by other Web sites, including those of your competition. So you can never go wrong by measuring your competitors’ online services and striving to match their performance. For example, Keynote publishes weekly indexes of Web site performance for several industry verticals, and many companies in these markets monitor these as an important benchmark of their site’s quality.

Some leading companies have taken this approach several steps further, setting up comprehensive measurement, tracking, and reporting programs for their most important Web initiatives, and letting their measurements of the competition drive their own service level objectives. One company even recalibrated the annual performance objectives and bonuses of IT and development staff, based on their ability to match competitors’ performance. And it works! The company improved from last to first place in its industry by following this SLM strategy consistently over a 3-year period.

To win at e-commerce, you cannot be content just to match your competition at every turn. Look at the situation from their point of view. Their customers’ expectations will also be based on the prevailing Web climate. And if your site is faster, then they can no longer delight their users with their responsiveness — the best they can do is satisfy them. This gives your site an edge in the usability stakes, which they now have to make up for by doing better in some other aspect.

So having a faster site gives you a competitive advantage. And if you are noticeably faster over an extended period, eventually some of their customers will become frustrated, and switch to your service.

Reporting Response Times

Once you have determined your response time objectives, how do you track and report them? First you must measure page download times. How you do this does not matter, provided that the measurements are a reasonable approximation of the response times that customers experience.

Next you must summarize and report these response time measurements. Unlike availability statistics, aggregate response time statistics are not self-explanatory and do not really show how well you are meeting your objectives. This presents a challenge, especially if you have set different response objectives for many pages of your Web applications. Until recently, there was no accepted way to reduce measurements to a common scale that would immediately show a manager the level of success being achieved through their SLM efforts. But recently the Apdex metric has come along.

Apdex, short for Application Performance Index, is a new open standard that seeks to address this problem. An alliance of companies whose business is measuring performance has defined a user satisfaction score (the Apdex metric) that can be easily derived from any set of response time measurements, once a response time goal has been set. The Apdex metric is a number between 0 and 1, where 0 means no users were satisfied, and 1 means all users were satisfied.

Summary

To be sure your site is fast enough, implement a systematic SLM program. That program must include measuring your competition, and using the results to set competitive goals. Then, after measuring the response times your own customers experience, you can apply the Apdex method to track your own success. If you apply these methods, you will be able to provide your customers with the responsiveness that satisfies them without spending unnecessary time and resources. You will know when your Web site is “fast enough” for your customers’ goals and your business goals.