Heartbleed: SaaS’ Forbidden Experiment?

Have you ever heard the term “The Forbidden Experiment”? If you’re not familiar with it, it’s a concept originating in the behavioral sciences relating to challenges in understanding human language development. Specifically, the “experiment” in question refers to actually testing empirically what would happen if a child were raised without language — i.e., if someone deliberately interfered with normal language development as a vehicle to learn how language development works and how a person might be different without it.

If this sounds terrible to you — that is, morally reprehensible — it should. In fact, the ethical and moral issues are why it’s described as “forbidden” in the first place. In other words, the consequences of doing empirical testing on language deprivation are so high — and the ramifications so terrible to the person who would be the subject of such an experiment — that as a practical course of action, it is off the table, despite the valuable information that might be gleaned.

This is in some ways conceptually similar to where the tech industry is with Heartbleed. Heartbleed, the issue that quite literally is taking the security profession by storm — and the consequences of which are, without hyperbole, likely to be felt for years — is something that nobody would voluntarily have signed up to have happen.

The consequences have been, are, and will continue to be terrible. That said, the fact that it did happen, and the fact that the scope of impact is as broad as it is, means that it’s a valuable learning opportunity. In particular, there are lessons for both Software as a Service customers and providers.

Lessons for SaaS Customers

For organizations that are SaaS customers — and let’s face it, most are — the Heartbleed incident is a valuable opportunity to learn something salient about your organizations and service delivery partners — specifically about their security posture.

Because it initially impacted about 17 percent of SaaS providers directly (according to data from NetCraft), the Heartbleed issue is significant enough that it’s reasonable to expect them to have some sort of coordinated response — even if that response is just to acknowledge the issue directly and inform customers clearly and unambiguously that they aren’t vulnerable.

This means that the Heartbleed issue can be a barometer of a given SaaS provider’s response and notification procedures. Certification programs (e.g., the CSA STAR registry) are useful as a proof point of a vendor’s security, but so too is knowledge gained through empiricism — i.e., how an organization responds in the face of the crisis to ensure that its service remains secure, and also what support it’s provided to trusted partners.

Reflect for a moment on how your key SaaS providers responded to Heartbleed. If they were vulnerable to the issue, did they quickly and transparently update their server certificate? Or did they drag their feet?

Whether they were vulnerable or not, did they notify you quickly, clearly, accurately and directly of their vulnerability status? How quickly did they do so? Was it done proactively, or did you have to nag them to find out?

How did that notification show up? Was it divulged in a direct and straightforward way, or was it hidden somewhere, so that you had to go hunting to find it — for example, in the fine print of a maintenance bulletin or an email to one person in the organization out of the loop of the broader coordinated response? The answers to these questions are all useful data points in evaluating their performance relative to security.

For the critical few SaaS suppliers in your organization’s scope, you probably know the answers to these questions off the top of your head, since the experience of their response is still fresh in your mind. Keep in mind, though, that a typical organization is likely to have many different SaaS providers. The answers to these questions are no less valuable for the providers where you might have a smaller usage footprint, so systematically reviewing their responses (i.e., from the vantage point of hindsight) is still productive.

Lessons for SaaS Providers

If you are a provider of SaaS services, there are lessons to be gleaned from Heartbleed in terms of how you deliver your services to customers. If your SaaS service relies upon SSL/TLS for secure communications with customers — and again, most do — then chances are high that customers of your service expected (or still expect) a response from you about Heartbleed’s impact relative to your services.

Think critically about how you handled the response to customers from a notification standpoint. How quickly were you able to respond to customer inquiries about the issue? Did you have a process in place to see the issue develop so you could proactively prepare a response to customers? Or did you have to wait for customers to tell you about the issue so that you could start investigating whether or not your service was vulnerable?

How were communications with customers handled? Was there a coordinated response to get consistent information into customers’ hands quickly? Or was there some period of ad hoc, inconsistent response through support channels? If you needed to replace a server certificate or needed to advise customers to change user passwords, did you do so quickly? How did that determination get handled — and were the right folks involved from the get go?

If you’re a service provider, there’s a temptation to see something like Heartbleed as all downside, but it’s an opportunity to shine as well. Customers notice when you are the one informing them about a potential security issue rather than vice versa. Customers don’t necessarily expect that your service be free of every vulnerability (even undiscovered ones), but they do expect that you take appropriate action, that you notify them about your status relative to the issue — without their having to chase you — and that you communicate clearly and directly.

The point is, now that it’s happened, there’s a lot that folks on both sides of the SaaS market can learn from Heartbleed. As Warren Buffet famously said, “It’s only when the tide goes out that you discover who’s been swimming naked.”