Putting CX at the Center of Testing Strategies

From e-commerce to banking applications to healthcare systems — and everything in between — if it’s digital, users expect it to work at every interaction, and on every possible platform and operating system.

However, despite the need to provide a digital experience that delights, Gartner research suggests that only 18 percent of companies are delivering their desired customer experience.

A big part of this gap between expectation and reality is that digital businesses depend on the quality of their software and applications, which frequently do not perform as they should. In an age when digital transformation is so dependent upon better quality software, testing has never been more critical. However, for the last decade, testing has focused on verification — that is, does it work? — rather than validation, meaning, does it do what I expect and want?

As companies progress in their digital transformation journeys, it’s critical that testing focus on answering the latter question. In other words, software testing must pivot from simply checking that an application meets technical requirements to ensuring that it delivers better user experiences and business outcomes.

Verification Validation

Testing needs to shift from a verification-driven activity to a continuous quality process. The goal is to understand how customer experiences and business outcomes are affected by the technical behavior of the application. More than this, though, it’s about identifying opportunities for improvements and predicting the business impact of those improvements.

Verification testing merely checks that the code complies with a specification provided by the business. These specifications are assumed to be perfect and completely replicate how users interact with and use the software.

However, there’s simply no way a specification writer could know how users will react to every part of the software or capture everything that could impact customer experience. Even if there were, it would make the software development painfully slow. By adopting this approach, the assumption is that validation also has been done as a result. However, this is a mirage rather than a reality, and has resulted in the customer experience being ignored from a software testing perspective.

Companies must abandon the outdated approach of testing only whether the software works, and instead embrace a strategy that evaluates the user perspective and delivers insights to optimize their experiences. If you care about your user experience and if you care about business outcomes, you need to be testing the product from the outside in, the way a user does. Only then can you truly evaluate the user experience.

A user-centric approach to testing ensures that user interface errors, bugs and performance issues are identified and addressed long before the application is live and has the chance to have a negative impact on the customer experience and, potentially, brand perception. Fast, reliable websites and applications increase engagement, deliver revenue, and drive positive business outcomes. Ensuring that these objectives are met should be an essential part of modern testing strategies.

For example, a banking app may meet all the specification criteria, but if it requires customers to add in their account details each time they want to access their account, they will lose patience quickly, stop using the app, and ultimately move to a competitor. This is exactly why businesses need to rethink how they evaluate software and applications and re-orient their focus to meet their customers’ expectations and needs.

If businesses want to close the customer experience gap, then they need to rethink how they evaluate software and applications. Validation testing should be a foundational element of testing strategies. However, organizations need to start testing the user experience and modernizing their approach so that they can keep up with the pace of DevOps and continuous delivery. This is an essential driver behind digital transformation.

Historically the only organizations carrying out validation testing have been teams with experienced manual exploratory testing capabilities. Exploratory testing evaluates functionality, performance and usability, and it takes into account the entire universe of tests. However, it’s not transparent, qualitative or replicable, and it’s difficult to include within a continuous development process. Manual exploratory testing is expensive to scale, as it’s time-consuming and the number of skilled testers is limited.

Customer-Driven Testing 101

Customer-driven testing is a new approach that automates exploratory testing for scalability and speed. Fundamentally, customer-driven testing focuses on the user experience rather than the specification. It also helps accelerate traditional specification-driven testing. Artificial intelligence (AI) and machine learning (ML) combined with model-based testing have unlocked the ability to carry out customer-driven testing.

The intelligent automation of software testing enables businesses to test and monitor the end-to-end digital user experience continuously; it analyzes apps and real data to auto-generate and execute user journeys. It then creates a model of the system and user journeys, and automatically generates test cases that provide robust coverage of the user experience — as well as of system performance and functionality.

Through automated feedback loops, you can zoom in on problems quickly and address them. Once that is in place, the intelligent automation can go even further — to where it builds the model itself by watching and understanding the system. It hunts for bugs looking at the app, the testing, and development to understand the risk.

It assesses production to clarify what matters to the business. This intelligence around risk factors and business impact direct the testing to focus in the right places. Unlike the mirage of testing to a specification, the actual customer journey drives the testing.

AI and ML technologies recommend the tests to execute, learning continuously and performing intelligent monitoring that can predict business impacts and enable development teams to fix issues before they occur. These cutting-edge technologies are core components of customer-driven testing, but another essential element is needed: human intelligence.

The Human Factor

Customer-driven testing doesn’t mean the death of the human tester. Machines are great at automating processes and correlating data but are not able to replicate the creative part of testing. This involves interpreting the data into actual human behavior and developing hypotheses about where problems are going to be.

The tester needs to provide hints and direction, as machines can’t replicate their experiences and intuition. Human creativity is essential to guide the customer-driven testing process.

Automated analytics and test products provide vast volumes of data about how a user behaved at the human-to-app interface, but it requires a human to understand why the person took that action. The human will set the thresholds for errors and will pull the levers and guide the algorithms, for example. Customer-driven testing is possible only with human testers augmented by state-of-the-art technology.

CX and the Path Forward

Digitization is rapidly changing the way companies and customers interact with each other. Understanding and optimizing the customer experience and ensuring apps deliver on business goals are now mission-critical for digital businesses. Practices that merely validate that software works must be retired, or organizations run the risk of lagging behind their competitors.

A new approach to testing is essential. The combination of AI-fueled testing coupled with human testers directing the automation makes customer-driven testing possible. If businesses want to close the customer experience gap, then they have to pivot and look at the performance of their digital products through the eyes of the customer. If software truly runs the world, then you need to make sure that it’s delighting your customers rather than merely working.