Improving Customer Experience: Usability Testing Is Not Enough

Many Web businesses do not provide a compelling customer experience; some sites are simply unusable, while others fail to provide content, goods and services that match their customers’ needs and expectations. To make matters worse, most e-tailers do a poor job of effectively assessing their customers’ experiences. They fail to learn why their customers behave as they do, a step vital to mapping any strategy for improvement.

To truly understand customers and gather the information necessary to make strategic business decisions, Web companies need to employ methods that yield richer customer insights.

Addressing Marketing Issues

Initially, Web sites were developed with more guidance from engineering groups and design agencies than from traditional marketing groups, and they were not subjected to the same degree of rigorous testing typically undertaken by marketing researchers.

User Interface designers are often buried in technical departments and have backgrounds in software development and graphic design. Unlike their marketing counterparts, they are not trained in product, pricing, placement and promotion. They often report directly to a product manager and are left out of strategic discussions altogether.

In many cases, the evaluation that sites receive is limited to functional issues. Although Interface Engineers sometimes conduct usability testing, the designers’ main focus is on whether users can achieve goals with ease. Thus, the metrics that usability evaluators tend to focus on have to do with the mechanical aspects of the site design.

Can people learn how to navigate the site easily? Are there too many graphics that slow down the site? Is there enough information on the site to insure that people complete forms accurately? Can people make a purchase without encountering problems? Although these are critical issues to address, they do not encompass the full scope of the user experience or measure the value delivered to customers.

Predicting Customer Behavior

In other cases, marketers may be put in charge of site design, despite their lack of knowledge concerning software development or usability issues. Focus groups may help them to uncover high-level conceptual insights. In many cases, though, this information is little help in determining how to design and implement a Web site to deliver what is desired.

Likewise, surveys may capture segmentation information and self-reported preferences, but they are incapable of capturing how people will actually behave on a site, or how the site experience will impact their brand perceptions and overall satisfaction.

More sophisticated research methods need to be employed if companies want to improve their understanding of how customers are experiencing a site. These methods must be applicable to both usability and marketing-research issues.

The ‘Why’ Behind Customer Behavior

The ideal customer-experience evaluation methodology captures an integrated blueprint of the user’s thoughts, feelings and behavior while interacting with the site. It addresses both marketing-research issues, such as expectations, preferences, satisfaction, pricing, positioning and segmentation, and usability-related issues, including ease of use, accessibility to different types of users, problems related to completing Web tasks, and navigation).

An integrated blueprint of the customer experience allows companies to better understand what they need to change to be more successful — they can understand whether they have a branding issue or an ease-of-use issue, and which is most important to fix first. Below are some of the common concerns that Web companies need to address.

Business Strategy Issues:

• Do users understand the site’s value proposition? Does their perception of the value proposition change after site usage?
• How does user experience on the site compare with competitors’ sites?
• After interacting with the site, are users likely to come back? Why or why not?
• Is the actual customer experience consistent with brand positioning? Is it consistent with the brick-and-mortar brand?
• What features are users expecting to see on the site?
• Are particular types of users (e.g. novice users, power users) reacting differently to the site? What special needs do particular groups have?
• Which aspects of the site experience have the most impact on overall success of the site and the company?

Design Issues:

• Can users easily accomplish critical tasks, such as searching and purchasing? If not, why not?
• What paths do users take in accomplishing critical tasks? What dead-ends do they encounter?
• At what point in the process of pursuing specific tasks do users typically fail or give up? Why?
• Do users notice and make use of particular features on the site?
• How much time and effort does it take to accomplish critical tasks? How can this best be reduced?
• Do users read and make use of information provided? Do users have enough information?

Data should capture behavior from a diverse group of users in their natural environments and in sample sizes large enough to make reliable projections. Qualitative comments should be collected simultaneously with measuring quantitative behavior so that users can explain why they acted as they did. All of these metrics should be analyzable at the site level to provide insight into strategic business issues, and at the individual level to provide insight into tactical design issues.

Integrated methods, such as those provided by Keynote Systems, have been developed to capture critical elements of usability testing and marketing research to assess the entire experience and explain the “why’s” behind customer behavior.

With the right data in hand, both marketers and designers can do their jobs better and work together more effectively to design products and services their customers value and to ensure satisfaction with the customer experience. Integrated customer-experience research methods are a critical tool that every business needs in order to win high-value customers and keep them coming back.