Cleversafe targets big data with 10 exabyte storage
Connecting state and local government leaders
Cleversafe debuts "dispersed storage" system capable of holding 10 exabytes of data, a scale required by new methods for analyzing big data streams.
The challenge of managing big data – from scientific research to social network analysis – will call for new ways to contain the digital outpouring.
Cleversafe, a Chicago-based developer of “dispersed storage” systems, introduced a technique Jan. 24 it says can hold 10 exabytes of data – the equivalent of a million 1 terabyte disk drives.
The firm said it its system is designed to hold data on a scale required by new methods for analyzing very large data streams over time. Potential applications include forensic analysis, scientific research, and commercial and economic analytics.
Related coverage:
Cleversafe Vice President Russ Kennedy said that with Internet traffic volumes increasing at a rate of 32 percent each year, “It’s not unrealistic to think companies looking to mine that data would need to effectively analyze 80 exabytes of data per month by 2015.”
Dispersed storage is a technology that splits data into slices, each of which is then stored to an independent storage appliance via TCP/IP. The technology’s “limitless data” capability is “a critical enabler to big data analytics,” said Kennedy.
The 10-exabyte system has been configured for scaling storage capacity and performance through a Portable Datacenter (PD), a collection of storage and network racks that can be easily deployed or moved.
Each PD contains 21 racks with 189 storage nodes per PD and 45 3TB drives per storage node. This geographically distributed PD model allows for rapid scaling and mobility, the company said.
Analysts believe big data will exert pressure on industry to develop other new approaches to storage management.
"The exponential growth of unstructured data has brought the storage industry to a turning point," said David Reinsel, IDC group vice president of storage. “In order for companies to continue to protect their data assets and to extract value from their vast vaults of stored data, they must begin looking at technology alternatives beyond RAID in order to scale without limits."
"Cleversafe's ability to deliver a system capable of storing exabyte levels of data today is a good example of the undertaking required to move forward and to solve the growing global issue of unstructured data,” he added.
NEXT STORY: DOD must trim, restructure network infrastructure