Big Data’s Big Federal Potential

Innovation often comes slowly to the U.S. government. It has taken several years, for example, for federal agencies to get used to cloud technology — and it will likely be several more before that technology is fully embraced at the federal level. That pace also seems to be the case for the federal adoption of Big Data and analytics, despite the significant amount of buzz over the past year.

Even in constrained times, however, federal spending on information technology is impressive, and just a small slice of that investment can be noteworthy. Vendors need to not only follow the money in terms of total federal IT spending, but also follow trends in spending objectives. It appears now that spending for Big Data and analytics is trending upwards and will continue at a modestly growing pace for the next several years, according to several recent forecasts.

Annual federal outlays for Big Data and related purposes will grow from US$5.22 billion in 2013 to $5.77 billion in 2018, predicted an October update on U.S. government IT spending issued by Deltek, for example.

That report breaks spending into two components, of which the storage segment will essentially remain flat during the period at around $3.5 billion per year. The remaining “core” Big Data/analytics spending, on the other hand, will amount to $1.73 billion in 2013. Spending will dip a little for a few years, but by 2018 annual core investment will jump to $2.25 billion. Core elements include high-powered computing, data services, software and data management.

Light Is Dawning on Data Potential

A separate report from IDC Government Insights issued last May showed a similar trend. That study pegged federal outlays for Big Data/analytics at $1.40 billion in 2016, compared with Deltek’s more recent estimate of $1.75 billion for the same year. Deltek includes estimates of spending by intelligence agencies and other government units not generally covered in IT budget tabulations.

Big Data was cited as one of three top investment areas for government agencies in the immixGroup’s federal forecast for 2014, released in late October. While the defense sector will lead the way, nondefense investments will still be significant, according to the forecast.

“As with defense agencies, civilian agencies are generating increasing amounts of structured and unstructured data in the course of day-to-day operations,” said Tomas O’Keefe, an immixGroup market intelligence senior analyst. “Business intelligence, analytics and knowledge management tools will all find use in improving business operations.”

Federal agencies are just beginning to comprehend both the potential and growing need for Big Data capabilities.

“These agencies recognize that extensive commercial computing infrastructure can be leveraged to provide large data sets and analytics to a wide variety of partners, customers and purposes,” Alex Rossino, principal research analyst at Deltek, told the E-Commerce Times. “Agencies know they cannot provide these services themselves, especially given the financial constraints they are facing, so they are leveraging commercial capabilities to get the job done.”

Yet the agencies will be compelled to justify such spending in terms of cost-benefit or return on investment calculations as part of the budgetary process. So far, the ROI element has been a somewhat elusive factor due to difficulties in clarifying objectives for investments. Not only that, the term “Big Data” remains so squishy that two British professors felt compelled to discuss the definition issue in a paper released only last September.

For purposes of federal procurement, an explanation from former Data.gov program director Jean Yan may suffice.

“Big Data appears to have become a buzzword overnight,” wrote Yan in an April 2013 report she undertook as part of a federal management program. “The term describes innovative techniques and technologies to capture, store, distribute, manage and analyze petabyte- or larger-sized data sets with high-velocity and diverse structures that conventional data management methods are incapable of handling,” added Yan, who currently serves as an education research analyst at the U.S. Department of Education.

Lack of Clarity Could Slow Adoption

“Confusion about how to define ROI is a potentially huge roadblock for expanding the use of Big Data tools in the federal government because agencies are instructed to write business cases justifying investments,” Rossino pointed out. “ROI estimates are at the heart of all business cases, so if ROI is difficult to define, much less calculate, it could deter agencies from making large Big Data investments.

“It is for this reason, among others, that we suspect agencies will continue to dabble in Big Data for isolated uses before adopting solutions more widely,” he said.

Cybersecurity could prove to be the exception, Rossino added, because as agencies develop cloud-based Big Data analytic capabilities to continuously monitor their networks for safety and security, the ROI is more easily determined.

“This makes investing in Big Data for cybersecurity purposes one of the most likely areas of growth we’ll see,” he added.

IDC’s research also indicates that federal agencies are taking a gradual approach. Thirty-five percent of responses to the company’s survey of 182 federal IT professionals said that Big Data/analytics activities are currently at the “research and consideration” stage, IDC reported, while another 6 to 7 percent are at the “pilot or proof of concept” stage.

Fewer than 10 percent of the responses claimed that Big Data initiatives were at the stage of “full deployment, IDC noted in its 2014 government predictions presentation released in early December. Less than 14 percent of the IDC survey responses said that analytics projects were fully up and running.

“We are seeing government move from discussing the “Vs” — volume, velocity, variety and value — to deploying Big Data and analytics capabilities that represent a mix of talent, technology and processes designed to economically extract value from very large amounts of data, or fast growing, multistructured data to support tactical, operational and strategic decision-making,” Adelaide O’Brien, IDC’s research director for smart government strategies, told the E-Commerce Times.

“Current survey data indicates that almost 75 percent of government IT decision-makers will consider Big Data solutions within the next 12 months, with only 26 percent indicating that this solution will not be considered in the coming year,” she said.

Support Services Get Attention

The pattern of spending for Big Data/analytics will tilt in favor of acquiring services, with lesser amounts for hardware and software. For 2013, outlays for services amounted to a shade over $1 billion, Deltek calculates, with software next at $430 million and hardware at $260 million. By 2018, outlays for services will be at $1.42 billion, with software at $540 million and hardware at $290 million.

IDC’s forecast shows service outlays amounting to almost 58 percent of Big Data/analytics spending in 2016, with hardware in second place at 25 percent followed by software at 17 percent.

The marketing implication for vendors is that in the area of data management, federal agencies will need more than a routine amount of hand-holding and customer service. Factors such as need, legislation and Office of Management and Budget directives are driving agencies to make better use of their data, the Deltek forecast concluded, but a lack of clarity on how to meet objectives “translates into agencies throwing darts at solutions.”

Vendors should monitor actions at the National Institute of Science and Technology related to the definition and range of what constitutes Big Data/analytics, since NIST guidance is often used in contracting.

Concluded Deltek, “industry must educate federal customers to shape demand and grow the business.”