Federal Investment in Big Data Applications Heads for Liftoff

U.S. government agencies appear to have gotten the memo: Big data is good for you.

Federal agencies’ acquisition, storage, processing and management of almost unimaginably large chunks of information will drive the government to use big data technologies, according to a recent survey of federal information technology managers.

In addition, the use of big data analytics to productively maximize the value of all this information will become a major goal of government agencies, the survey showed.

To accomplish those goals, federal IT managers increasingly are seeking the support of the private sector.

Forty-six percent of respondents planned to increase use of third-party contractors or consultants to assist with big data projects, according to the survey, which was sponsored byUnisys Federal. Another 52 percent intended to maintain their current level of engagement with outside providers.

Support Components Are Lacking

A major reason agencies are seeking private sector support for big data capabilities is that these agencies have a talent deficit.

“Government agencies are having a difficult time in hiring and retaining data scientists,” said Rod Fontecilla, vice president for advanced data analytics at Unisys Federal.

“The survey also found that many agencies don’t have the IT infrastructure to support advanced data analytics. So third-party companies can bring together the human and IT resources, such as data scientists, needed to help the agencies meet their challenges,” he told the E-Commerce Times.

Federal investment in big data has been relatively modest, as well as uneven, in recent years, according to an analysis byDeltek, which tracks government IT spending.

Modest, of course, is a relative term when hundreds of millions of dollars are involved, but in 2012 federal spending for big data amounted to about $832 million. Investment dipped to $693 million in 2013 before rebounding to $1.0 billion in 2014, the latest full year tracked by Deltek.

Those investments were affected by the budget reduction sequestration process in 2013. However, the 22 percent gain between 2012 and 2014 was notable, according to Deltek.

Future big data spending could increase at a compound annual growth rate approaching 9 percent, presuming that budgets follow their normal course and are not subject to stop-and-go congressional machinations.

“We still believe that this is an area of rich opportunity provided that the fiscal situation does not become too constrained,” said Alex Rossino, senior principal analyst at Deltek.

Annual Investments will Reach $4 billion

The amount of big data spending and the growth rate of investment really will start to move upward in 2016, according to a market analysis fromimmixGroup, an IT distributor in the public sector.

In 2016, civilian big data spending could reach nearly $2 billion, while the level in the defense sector likely will hit $1.6 billion for an annual total of $3.6 billion. The total could reach $3.9 billion by 2017 and $4.2 billion by 2018, according to immixGroup projections.

“We anticipate about an 8 percent CAGR between 2016 and 2019 across the government,” said Stephanie Meloni, a senior analyst at immixGroup.

“That’s going to vary by agency,”she told the E-Commerce Times.

A spending projection from Deltek is a bit more modest, although rate of growth for federal big data investments also is impressive.

In 2016, federal agencies will spend about $1.7 billion for big data projects exclusive of storage, of which $1.07 billion will be for services, $440 million for software and $250 million for hardware. By 2019, the total will be $2.4 billion, with $1.54 billion of that for services, $610 million for software and $290 million for hardware.

The Unisys Federal survey of 100 federal IT managers was conducted byBeacon Technology Partners, and results were released in early October.

Overall, these managers were surprisingly supportive of the potential for big data technology. Almost 50 percent of survey respondents had a favorable view of big data analytics as a vital tool for agency operations. Only 21 percent expressed a negative or somewhat unfavorable view, while a third of respondents registered a neutral position on the value of the technology.

However, agencies generally are finding it difficult to implement big data programs, with only 16 percent of respondents reporting that their agencies had launched a project, while another 38 percent said their agencies were engaged in pilot projects. The remaining 46 percent registered intentions to implement or explore projects within the next 24 months.

For vendors, the survey not only indicated that the potential government investment in big data will be significant, but also that much of that investment will be directed to the private sector. In addition, the survey provided key insights into the areas where federal agencies likely will allocate big data-related investments.

Survey Reveals Spending Priorities

For example, asked about the types of data agencies will target for big data projects within two years, respondents put structured data at the top of the list at 40 percent. Real-time data was next at 23 percent, followed by semistructured data at 20 percent and unstructured data at 18 percent.

Given various sources of data that will be subject to such investments, the top-ranked items survey respondents cited included sensors and machine-generated data at 63 percent; telecommunications call data records, also at 63 percent; geographic and geospatial data at 59 percent; and web-logs at 55 percent.

Investments in IT support operations will be required to facilitate big data projects. The top items for such investments federal IT managers mentioned included increasing network bandwidth at 63 percent, more robust data security at 55 percent, cloud-based analytic services at 54 percent, and more agile and flexible network architecture at 50 percent, survey results showed.

The link between innovative IT infrastructure technologies and the development of big data capabilities will be a significant developmental factor.

“Many government agencies still are reluctant to use the cloud, but that is slowly changing as the cloud is being used more and more to address critical problems,” said Unisys Federal’s Fontecilla.

“Big data analytics is going to play an important role in cloud adoption, because most organizations don’t have the infrastructure in-house to support their need for advanced data analytics. Furthermore, finding data scientists who understand how to use machine learning algorithms to create predictive models is very difficult,” he said.

“We are seeing more and more RFIs and RFPs from government agencies requesting either the creation of a big data analytics environment or the use of machine learning algorithms to help government agencies make better business decisions,” Fontecilla said.

Strategic Approach Will Spur Activity

Based on the analysis of investment information through 2014, Deltek’s Rossino sees a similar trend.

“As of the data we had last year, we saw two areas where vendors could find opportunity: providing technical support for big data programs and developing big data capabilities for agencies,” he told the E-Commerce Times.

Still, maximizing the full potential of big data will involve a broader appreciation of the technology as a component of a more holistic approach to IT management, rather than as the next new thing in technology.

Many agencies are starting to recognize the challenges related to big data implementation and taking steps to address them, Rossino said. These measures include developing data management and digital strategies, hiring chief data officers, and launching pilots to develop experience with the technology.

“What most agencies still fail to realize, however, is that big data is best viewed as part of a business management strategy,” he said.

“It is only when agency leadership buys into the value that big data can provide to improve business processes that it will be more widely adopted,” Rossino said. “Until that point is reached, big data implementations will continue to be isolated.”