Big Money for Big Data

Accel Partners is launching a US$100 million fund to identify — and fund — entrepreneurs at every layer of the big data stack, in particular, those developing projects for the Apache Hadoop platform.

The Accel Big Data Fund is aligned with Cloudera, which is pushing development in Hadoop.

These products could be anything from next-generation storage and data-management platforms to software applications and services, including data analytics, as well as vertical and mobile applications.

Accel Partners will manage the fund from its offices in the U.S., London, China and India. It is also planning a big data conference in Silicon Valley for Spring 2012.

Accel Partners did not respond to the E-Commerce Times’ request for further details.

Data Proliferation

The enterprise has created an unprecedented volume of data, especially unstructured data, in recent years, and the rate of its creation is only expected to increase.

“We keep adding data,” Richard McCarthy, associate dean of the School of Business and professor of computer information systems at Quinnipiac University, told the E-Commerce Times.

“We never delete or archive it,” he noted. “Huge datasets, such as Google Earth or Walmart’s data warehouse, have become difficult to access and maintain using current technology. Databases have gotten so big that we need to develop the next generation of technology that can capture, store, analyze and report on datasets that are measured in quadrillions of bytes.”

While the concept of big data is exciting, there are mundane factors that must be addressed as products are developed, Paul French, VP of product and solutions marketing at Axway, told the E-Commerce Times.

These include where the files are coming from, security, and visibility into those files, along with timeliness and accuracy of the data. The big data needs of trading partner relationships are another element to consider, he added.

Small Companies vs. IBMs of the World

The IT community has hardly been idle in this area, Charles King, principal of Pund-IT, told the E-Commerce Times.

“We are seeing large vendors like IBM and EMC and even HP working in this space,” he noted. “Oracle too, although its position is a little cloudier. What they all have in common, though, is that they are trying to pitch big data as the extension of traditional database products.”

Big data solutions don’t necessarily fit within the confines of a relational database, however, he noted.

That is why much of the innovation in this space is coming from open source and from startups.

“They aren’t constrained with worries about cannibalizing better-established product lines,” King said.

The Healthcare IT Example

One sector where small firms are particularly active in developing such products is in healthcare IT, he said. That is because unstructured data — medical imaging and so forth — is rampant.

“The smaller players are playing with new ways of storing medical images, as one example, and they are introducing very innovative solutions,” King said.

This is how other big data sectors will likely evolve — via smaller, tactical solutions developed by startups. “The larger, overarching analytical approaches are already covered by the IBMs of the world,” he concluded.

“Small companies have the luxury of taking big risks and tackling impossible problems from entirely new angles,” observed Yaron Samid, cofounder and CEO of BillGuard.

” We are not bound by the norms of corporate inertia and can therefore innovate — and fail — much faster,” he told the E-Commerce Times.

“Normalizing massive inconsistently structured and unstructured data sets from across the Web, for example, is a hairy problem that requires new tools, new methodologies and new thinking,” explained Samid. “The bleeding edge is taking shape at startups, not large corporations.”