Keynote’s Douglas van Duyne: Are You Customer Experienced?

In an effort to further expand the breadth of its offerings, Web performance measurement firm Keynote Systems recently acquired NetRaker, a developer of customer experience management software. In other words, NetRaker’s software is designed to help e-business managers measure and improve the navigability of their Web sites.

Most of NetRaker’s employees — including president and CEO Douglas van Duyne and COO George Papazian — joined Keynote. Van Duyne, who, with James Landay and Jason Hong, co-wrote a book called The Design of Sites, is now senior director of customer experience products and designs at San Mateo, California-based Keynote, while Papazian was named general manager of the customer experience division.

Keynote did not disclose details of the acquisition, which closed in April, but predicts that NetRaker’s products and services — including Research Manager and NetRaker Index — will boost third-quarter revenue by about US$500,000.

The E-Commerce Times recently had the opportunity to speak with van Duyne about NetRaker, the acquisition and the vital role customer experience management plays in the success — or failure — of any e-business.

E-Commerce Times: Can you tell us a little about NetRaker?

Douglas van Duyne: I founded the company in 1998 with professionals in computer-human interaction. I had been on the practitioner side. I had been doing work for Safeway, for Intel, [and] I found customers were running into the same problems over and over again. The Web was new territory where people were not coming to it with a lot of experience in design.

That was one reason we authored the book — Design of Sites. We really set out to help site developers [and] site managers understand the importance of customer-centric design and the customer experience. If you think about the management of sites, it requires a feedback loop between the customers and designers of the site. We feel this involves people from many factions: usability research, PhDs in computer-human interaction, market research. Customer experience management is a combination of many different [abilities].

ECT: So that’s the synergy between NetRaker’s emphasis on measuring the customer experience and Keynote’s emphasis on Web performance.

Van Duyne: It’s a really strong value proposition. The combination can really improve the efficiency and value of [a customer’s] Web site. It broadens from the technology focus that existed at Keynote into the marketing side. We’ve become a one-stop shop for user-experience services.

It provides the ability to answer questions about the scalability of the performance characteristics of a site from the IT perspective, as well as some basic continuity management and analysis issues. We provide load testing, transaction testing, performance testing. There’s a performance scoreboard. There’s a network performance metric. And it measures the quality of the content: Does this site have what people are looking for? That’s what’s first and foremost on users’ minds.

A site can add to a brand, but if the site has a shoddy design or provides a poor experience, that can really detract from the brand. That didn’t happen before. You wouldn’t need to build an interface for a consumer company.

ECT: What about different demographics?

Van Duyne: We’re really looking for a combination of best practices and market research, usability and Web analytics. So, think about what people have been doing in the past, prior to the Web, to help companies understand their customers. Market research included focus groups, surveys, telephone research and product concept all the way through satisfaction: Who are my customers? Who’s doing what? Major companies are very familiar with that and very used to using market research to ground their marketing decisions.

Using research in the past typically has been the purview of engineering departments at software companies. There aren’t a lot of companies selling software out there. That discussion has been a focused discussion, but only at certain types of companies.

Now, as companies are evolving and really putting an [online] interface to their companies — sophisticated, meeting consumer needs, yet easy to use — we need to train a whole new breed of people to do this. Companies had built their Web sites in organic ways and had not built them from the customer perspective.

ECT: Where would you say the market stands today?

Van Duyne: The majority of companies are still in that evolutionary process, and a small group of companies have not even started that evolution. Web analytics [is] something companies have been relying on to find out which Web pages people are using, but it’s been hard for companies to use that information. It’s a morass of data. It also doesn’t have intention data — did someone intend to end up where they did? Were they planning on researching a product, then buying it offline?

We’re able to combine the capacities that have traditionally come from different disciplines to create what we call the ‘complete user experience.’

ECT: And how do Keynote and its customer experience group do that?

Van Duyne: We have specific services that address areas such as potential customers to a site [and] actual customers to a site. Research Manager allows you to do directed testing — to create, implement and analyze usability and market-research studies. Real User Observer allows you to test what real users see when they visit your site.

ECT: So it actually sees what people are doing online at a company’s site?

Van Duyne: Yes. Who are you? Are you visiting Walmart.com to see what they have in the store, or to shop online? Knowing whether [a user] came to shop online or to browse — if you make an assumption that people are checking out before the checkout process — is critical to be able to capture what their intent is. This enables us to see who’s coming to the site, what they’re trying to do and what they are doing. Our service also allows us to drill down to the page level to see what they’re doing on a page-by-page basis: It gives quantitative numbers to back up qualitative findings.

Critical issues have been shown to be continually found as you add more and more participants. You could spend a lot of time and money fixing problems that were problems for five or 10 people, but never know if these were problems for more people. The Real User Observer is our service to unobtrusively track people visiting a site. Our Customer Experience Benchmark gives companies a way to keep their finger on the performance pulse of the site. It lets them drill down to what’s broken and shows them how to fix it.

ECT: How about cost?

Van Duyne: It’s a la carte on a subscription basis. Companies are engaged in annual subscriptions, multi-year subscriptions to monitor their sites. Companies like Cisco are running, literally, hundreds of studies. It’s really across industries — Eastman Kodak, Expedia, Travelocity, e-commerce companies of many different types.

ECT: What are the benefits?

Van Duyne: It really goes to, what are the critical factors for Web site success? Really, the answer is the quality of the service. If Google, for example, delivered great results but was hard to use, then it would become less valuable. It is tied very critically to the customer loyalty picture. If you’ve got quality content but people can’t find it, then they don’t know it’s there.

Companies evolving on the Web have a very strong practice in usability results or customer-experience management. Their focus on the customer and on building what the customer’s looking for makes them the best-of-breed of the dot-coms. They have a habit and practice of research.

ECT: How do Web site performance and the customer experience impact brand loyalty?

Van Duyne: If the system is easy to use, it improves the brand. If a company is trustworthy before the Web, there’s something already brought to the table. Having strong elements in each of those is the best ideal. There are factors people will give on and still succeed. A great example is fashion and apparel: There’s some rich content on those sites. People are more willing to tolerate a slower-performing site because of that. People are willing to put up with a longer download if they can get the info and graphics they want.

ECT: Are there any rules or guidelines you would give e-commerce managers?

Van Duyne: Really, there are a number of golden rules. They must test the site’s concept and design as early as possible and as frequently as possible to ensure it’s the right thing being built and it’s being built the right way. When you test early and frequently, you reduce the cost — sometimes [by] as [much] as a factor of 10. There are a lot of downstream costs after a design is complete — programming, maintenance — that you can really reduce by thinking ahead.

Even when it comes to a site redesign, testing new designs early and frequently before they get implemented results in lower downstream costs.

As part of that process, working with a customer-service task force, if there isn’t already an existing organizational structure that does that, is vital. They must understand what people are looking for instead of being focused on granular interface issues. This [task force] should include individual business managers that are running different verticals in a Web site. It should include business managers working with researchers. Many times, those experts are people who need to be brought in, [because] many times, companies grew their site development staff organically. Many times, they brought in print designers without interactive design experience.

My last golden rule would be to learn from the best practices on the Web.

ECT: How aware is the market of the need for customer experience measurement and analysis?

Van Duyne: Evolution doesn’t happen overnight. There are some that have evolved all the way. The majority of service companies are already there. Others are in the process of going through that. We’re certainly out there evangelizing. Part of it is purely an organizational issue.

ECT: Any examples of the good, the bad and the ugly?

Van Duyne: There are some stark examples. They’re almost comical at this point. Boo.com was a $500 million venture capital company that blew through money. They didn’t look at the critical factors. They didn’t test frequently. They didn’t run research, so they didn’t know the site was bad until they launched it.

There are existing, very profitable offline companies that are still struggling with this. Some retailers are struggling. Companies that have been trying to stay with the best-performing sites on the Web are having a tough time.

ECT: That sounds like the bad. How about the good?

Van Duyne: I would single out Yahoo, Google, Expedia and others. There are companies that have traditional businesses that are also doing very well online, companies like Kodak.

On the ugly side, there are still companies that are blatantly ignoring their customers. Their online businesses are suffering as a result.