Keynote Systems Ranks E-Tailers on Speed, Reliability

Keynote Systems on Wednesday announced new indices tracking best-in-class e-commerce sites. The new indices track the speed and reliability from the end user perspective of 24 leading online retail sites across three categories: apparel, books and music, and electronics.

Conducted by Keynote Competitive Research, the industry analysis group of Keynote Systems, the new Keynote Online Retail Transaction Performance Indices replace the company’s single e-commerce index. The new indices are designed to provide the market with a deeper, more comprehensive window into the technical performance of online retailing.

“Web site performance is vital to the success of a company’s online sales — and the leading e-commerce retailers being monitored by Keynote Systems must do everything they can to ensure their sites are operating at a maximum level of performance 24/7,” advised E-Commerce Times Publisher Ric Kern.

The indices are meant to reflect the industry’s increasing breadth and importance to the overall U.S. economy. Online retailing will reach US$108.5 billion in the U.S. in 2006, according to eMarketer. The new retail indices serve as internal and external performance benchmarks for judging online success.

“Our indices show that a large gap remains between the best-in-class sites and sites that still have room for improvement,” Ben Rushlo, senior manager of competitive research with Keynote, told the E-Commerce Times. “Best-in-class sites rank well in overall performance and availability. We’ve noticed that electronics sites rank well in availability, but apparel sites struggle in that area.”

Critical Metrics

Because search engine optimization and search engine marketing initiatives play such an integral role in online retail traffic generation strategies, one Keynote discovery is especially troubling: Many errors captured in its preliminary data were with the search process or with adding products to online shopping carts.

“This illustrates the need for a multi-step transactional index as opposed to simply measuring the site’s home page — a page that typically is highly available,” Rushlo noted.

Another critical metric in the Keynote Online Transaction Performance Indices is the number of outage hours. This metric, which is new to the indices, indicates that a significant number of users from various geographic locations were unable to complete the measured path — shopping and adding a product to the cart.

“Ideally, a site should have not a single outage during the week, and many of the sites do achieve this goal,” Rushlo noted. “Other sites, however, consistently struggle to not have at least one major negative event during the week.”

Breaking Down Categories

In general, the Keynote Online Transaction Performance Indices show that performance was best for the electronics sites and poorest for the books and music sites. The overall performance metrics are a summary of the total download time of all the pages measured on the site.

“It is equally important to understand the variation in performance — both over the day, which indicates the site’s ability to handle load, and over the week, which is a good marker for overall site health,” Rushlo said.

Keynote’s Retail Apparel Index for the week of October 9 ranked online shoe retailer Zappos No. 1 in both speed and reliability with a response time of 3.83 seconds and a 99.87 percent success rate. JC Penney ranked second in both categories with a 4.49-second response time and 99.74 percent success rate.

Kohl’s, Banana Republic, Macy’s, Nordstrom, Eddie Bauer, Neiman Marcus and the Gap were also among the top nine in reliability and speed. Interestingly, each company ranked in both categories. The Gap posted four outage hours during the week.

Ranking Books and Music

In the books and music category, Wal-Mart took the top spot in terms of response time, posting 4.19 seconds. Barnes & Noble ranked first in reliability with a 99.87 percent success rate.

Overstock.com, Target, Borders, Buy.com, Tower Records and Amazon.com rounded out the top eight e-retailers in this category. Amazon was one of the slowest in the pack, with a 10.82 percent response time. Overstock.com, Tower Records and Buy.com each posted two outage hours for the week.

The Retail Electronics Weekly Report shows Office Depot has the fastest response time, with 3.96 seconds. CDW, Circuit City and Staples offered a 100 percent success rate.

Wal-Mart, Amazon.com, Buy.com and Overstock.com also ranked in the top 10 in the Electronics category, along with Dell and Best Buy. Amazon had the longest response time of the group, at 12.02 seconds, while Office Depot posted three outage hours for the week.

Accessing Weekly Data

Keynote reports an improvement in overall performance results over historic data. The company measures computers in 10 cities around the U.S. clicking through the same steps an actual person might perform, such as entering the particular URL of the Web site, searching for a product, filtering by category, drilling into product detail and adding the product to the cart.

The aggregated results of the new indices will be available each Tuesday on Keynote’s Web site and via the E-Commerce Times atwww.ecommercetimes.com/web-performance/.

“We believe that the ultimate result of online retailers competing for the top spots on Keynote’s performance rankings will be an overall improvement in the consumer’s online experience,” Kern remarked. “As we begin the fourth consecutive year of publishing Keynote’s Web Transaction results, the E-Commerce Times is looking forward to seeing even greater benefits to the online consumer from this initiative.”

The new retail indices, including all data points captured by Keynote, are available for purchase, both in bulk form and in a management level report. Prices begin at $3,000 per month per index.