White House Pulls Back the Curtains on Big Data Project

The U.S. government is awash in data that is collected, stored and disseminated by federal agencies to fulfill their public service missions. However, the existence of huge amounts of information within the government presents a major profit-making opportunity for commercial firms.

The Obama administration has now launched a program designed to make the data resources of the federal government much more available to the public and to private firms for commercial development.

By executive order, President Obama issued a mandate to federal agencies that “new and modernized government information resources,” should be both open to the public and available in “machine readable” form. According to the order, government information should be viewed as an asset throughout its life cycle and regarded in a way that promotes interoperability and openness. Information should be released to the public in ways that make the data easy to find, access, and use, the order said.

The mandate will significantly expand the amount of government information that is already available to the public on a voluntary basis.

President Cites ‘Amazing’ Advances

The scope of the program not only includes newly generated data, but also existing resources that can be upgraded to modern systems of access. The initiative should eventually provide business opportunities for the private sector, similar to the opening of federal data for Global Positioning Services (GPS).

“One of the things we’re doing to fuel more private sector innovation and discovery is to make vast amounts of America’s data open and easy to access for the first time in history. And talented entrepreneurs are doing some pretty amazing things with it,” said President Obama when the White House issued the order on May 9.

“Starting today, we’re making even more government data available online, which will help launch even more new startups. And we’re making it easier for people to find the data and use it, so that entrepreneurs can build products and services we haven’t even imagined yet,” he said.

The federal Open Data initiative should become a major source by itself for commercial activities, and a hefty contributor to the Big Data market, valued at about US$11-billion in 2012, according to Wikibon.

Commercial Sector Welcomes Program

“The Open Data policy dovetails well with the emergence of managing Big Data, and it will present a great opportunity for traditional hardware and software providers,” Chris Wilson, vice president, federal government affairs, at TechAmerica, told the E-Commerce Times.

“This new policy may increase commercial opportunities for any kind of enterprise whose work is fueled by government data, especially since it represents a new, fundamental commitment to releasing new data,”John Wonderlich, policy director at the Sunlight Foundation, told the E-Commerce Times. The policy should also improve public transparency of government agencies, one of the prime objectives of the foundation.

While the Open Data policy holds potentially significant benefits for the commercial IT community, the shape of that future is a bit sketchy right now.

“It’s unclear to me how traditional hardware and software companies will fare in the new ‘open by default’ information architecture. Decoupling data from its proprietary business system and enabling it to be reused freely, with little or no friction, is a novel concept. Many business models were built on a different set of assumptions,” Saf Rabah, vice president of marketing at Socrata, told the E-Commerce Times.

“But what is clear, and exciting, is that citizens will see enormous benefits in terms of enhanced access to information and innovative services that will improve their daily lives,” he said. Socrata provides cloud-based and other data management technology services to local, state and federal governments.

In addition to providing opportunities within the IT community, the Open Data policy is designed to spur commercial innovation from specialists in information content, including statisticians and others with subject matter expertise.

“It’s hard to quantify the opportunities that can result from the Open Data policy for data content specialists or entrepreneurs with ideas for utilizing federal information,” Wilson said. “But it’s almost a case of ‘build it and they will come.’ The more that government data is accessible the more likely it will be for innovators to find profitable uses for it,” he said.

Agencies Must Move Quickly

The Open Data order was accompanied by an implementation directive issued by the Office of Management and Budget (OMB). In combination, the documents direct federal agencies to:

• Collect or create information in a way that supports downstream information processing and dissemination activities;
• Build information systems to support interoperability and accessibility;
• Improve data management and release practices;
• Strengthen privacy and security measures.

Federal agencies must start implementing the Open Data directives in relatively short order. Within 30 days the federal chief information officer and chief technology officer are required to publish an open online repository of tools and best practices to assist agencies in meeting Open Data objectives.

By early August, federal acquisition specialists must identify and begin implementing measures to support Open Data in federal procurement and grant-making processes. Such efforts may include developing sample requirements language, grant and contract terms, and workforce tools for federal personnel involved in acquisition, grants, and IT functions. Within six months agencies will be required to report on their progress in meeting Open Data goals.

To draw attention to the potential use of federal information, the administration has already collaborated with the private and nonprofit sectors through a series of White House ‘datajams’ and ‘datapaloozas’ attended by government and non-government parties. Topics have included information issues related to energy, education and health care.

“As a result, there are private companies using open data to fight credit card fraud; consumers using Open Data to save on their energy bills; families leveraging Open Data to compare health care options; and a host of new apps and tools in areas ranging from public safety, to financial planning, to education, and more,” said federal chief information officer Steve VanRoekel.

However, these efforts are just the tip of the iceberg. “Many more government datasets are still hard to find or are locked up in unusable formats,” said VanRoekel. The government is sitting on a “treasure trove” of information that could spur significant economic growth, he added.

While most of the attention on Open Data involves the potential value of accessing and distributing economic, scientific and other chunks of federal information resources, the Data Transparency Coalition (DTC) has spotted another result benefitting business.

Regulatory Compliance Impact

“The scope of this directive also involves federal information related to regulatory compliance, especially in the financial sector,” Hudson Hollister, executive director of the DTC, told the E-Commerce Times. “This executive order and accompanying Open Data policy will cut compliance costs by allowing the tech industry to automate reporting and compliance processes that are currently manual.”

“For example, the Securities and Exchange Commission and other financial regulators have tons of reporting requirements and forms that are costly and time consuming,” he said. The interoperability, formatting upgrades and machine readable requirements of the Open Data initiative should lead to innovations and improvements in document compliance. “Theoretically some innovator could come along and develop a Turbo-Tax compliance program for submitting financial data to the government.

“Even our capital markets will benefit because public regulatory filings converted into Open Data will become a more accessible source of actionable information for investors,” he said.

“By making Open Data a requirement for federal agencies instead of a discretionary element, President Obama has handed the U.S. technology industry a key to expand our global leadership in this era of Big Data,” said Kevin Richards, senior vice president of federal government affairs at TechAmerica.