Proceed With Caution: Big Data Requires Small Steps

The Obama administration’s support of Big Data technology for more efficient and effective government operations was highlighted at a recent White House sponsored conference. Billed as the “Data to Knowledge to Action” event, the Nov. 12 session included presentations on more than 30 public and private sector projects successfully employing Big Data technologies.

“Realizing the enormous potential of data requires a bold, comprehensive, national effort, with engagement of stakeholders from all sectors,” Farnam Jahanian, assistant director for computer and information science and engineering at the National Science Foundation, said at the event. “These partnerships lay the groundwork for new discoveries, infrastructure and education opportunities that will comprise the foundations of U.S. competitiveness for decades to come,” he added.

While there may be a great potential payoff for investments in technologies to manage and analyze quantum leaps in data and information, a report from a public policy think tank offered a temporizing view on investing in Big Data projects by federal agencies.

The report, “From Data to Decisions III,” by the Partnership for Public Service, was released almost simultaneously with the White House event.

Tight Budgets Require Justifications

One of the key points of the report is that government agencies will need to better demonstrate the realistic payoff for making an investment in a Big Data project. Such investments will require improved justifications not only on their own merit, but also in light of constrained federal budgets in coming years.

In times past — say 20 or more years ago — before the emergence of the cloud and other IT innovations, government data projects largely bubbled up from mid-level ranks of staffers who were seeking to solve a fairly specific problem related to the direct mission of an agency or program. Thus the objective — or payoff — was clearly defined, and the investment was commensurate with achieving the objective, the report notes.

Following advances in IT over the last decade or so, there is now a tendency for major projects to be directed from the top down at federal departments and agencies, for more generalized or even vaguely defined purposes simply to employ available IT innovations.

“Today managers are tempted to begin analytics programs before determining the mission-essential questions they are seeking data to answer. This is possible because computers and software now can store and analyze data faster and at less cost than ever before,” the report notes.

In the current and future environment, agencies need to generate return on investment or cost-benefit calculations in measurable monetary values for data management and analysis projects, the report advises.

Strategic Approach Beats Formula

However, the Partnership for Public Service report does not offer any type of standard calculation that federal agencies can use to determine an acceptable cost-benefit ratio or ROI. In fact, the variety of agency missions and programs may work against the creation of a uniform formula.

“I like to frame the ROI question in terms of programmatic ROI, which is a more valuable exercise appropriate for Big Data projects. The key is not to focus on the use of Big Data per se, but on using Big Data as a tool to accomplish program objectives,” Brian Murrow, associate partner at IBM global business services, told the E-Commerce Times. The IBM Center for the Business of Government is a cosponsor of the report, and Murrow was a member of the team that prepared it.

“Often the first thing we do with clients, especially government agencies, is get them to clarify what the objective is, since so many federal programs lack specificity,” he said.

“The next step is getting the agencies to answer the question, ‘How are we doing now in meeting the objective?’ so you get a baseline. That allows you to measure any future improvements or achievements,” said Murrow.

“If the project is something new and has no budget, then you try to maximize the program objective at the least cost,” he suggested.

“If there is a budget, then you try to maximize program performance within the budget. I don’t know if there is a one-size-fits-all metric for ROI on Big Data for the entire government, since program objectives are so different — but there is definitely a methodology that can apply to all agencies if you take a strategic approach,” Murrow said.

“ROI is by far and away the most difficult success metric to determine for Big Data investments,” said Alex Rossino, principal research analyst at Deltek.

“Multiple questions abound: Will the investment save money, save lives, make operations more efficient, cure disease, improve air safety? Any of these metrics could be considered ROI,” he told the E-Commerce Times, “but will that be acceptable to the Office of Management and Budget?”

Calculating improved outcomes — such as increased numbers of food-borne illness outbreaks detected or enemy combatants identified — is an essential factor for mission analytics programs, the report notes.

“But just reporting better outcomes is not sufficient, especially now that sequestration is compelling programs to compete fiercely for scarce dollars. Agency leaders need cost-benefit metrics and measures of ROI to prove that data-based efforts compare favorably with other programs during budget reviews. So don’t focus on core analysis so single-mindedly that you fail to develop data to demonstrate ROI,” it advises.

Agencies Struggle With Investment Issue

There is a worrisome knowledge gap among government professionals on how to judge the value of investments in business analytics, IDC Government Insights found in a report that also touches on the issue of productive Big Data investments. The report, “Big Data for Government: Better Mission Outcomes by Unlocking the Hidden Value of Information,” was released last May.

Investments in business analytics projects paid off for about 30 percent of 182 survey respondents, but for 17 percent, they did not. Significantly, more than half were in the dark on how to gauge a value. About 10 percent of respondents said they simply did not know if the investment was of value and — significantly — 43 percent of respondents said they were “not sure how to measure the benefits of business analytics.”

Federal agencies approach Big Data projects on a solid strategic basis similar to the approach offered by IBM’s Murrow, IDC suggested.

“We advise government CIOs and other C-level executives to identify their needs and set the strategy for using information and technologies such as Big Data analytics; identify expected mission outcomes; and build organizational support for analytics in support of data-driven decision making from pilot projects that have borne fruit — and measure those outcomes,” Adelaide O’Brien, IDC’s research director for Smart Government Strategies, told the E-Commerce Times.

“If and when government program budgets get funded based on ROI,” she said, “having measurable outcomes will ensure that the programs with the biggest impact get funded.”