Speed vs capacity: Fermilab pilots solid-state storage
Physicists at the Fermi National Accelerator Laboratory are experimenting with nonvolatile memory express (NVMe) solid-state technology to find the best way to access data from the CMS particle detector at CERN’s Large Hadron Collider.
The Large Hadron Collider generates 40 terabytes of data per second on 40 million events. Much of that data is automatically filtered, keeping just data just on the collisions the scientists might consider investigating, resulting in only about 1,000 event per second being stored.
Still, tens of thousands of terabytes of data from the CMS detector have to be stored and backed up each year. Forty percent of those backups are stored at Fermilab.
“We have racks full of servers that have hard drives on them, and they are the primary storage medium that scientists are actually reading and writing data to and from,” said Bo Jayatilaka, the Fermilab scientist who is leading the NVMe project. The hard drives, however, are limited in how much data they can load into applications in a given time.
Until recently, that has not been much of a problem because high-performance computers retrieve data from storage sequentially, a task well suited to hard drives. Now, though, scientists are increasingly deploying machine learning in their work. These algorithms repeatedly access the same piece of data, and when thousands of processors are trying to access one data point on a traditional hard drive simultaneously, bottlenecks occur and processing slows.
Scientists at Fermilab recently installed a single rack of the solid-state servers its Feynman Computing Center to explore how to deploy the new solid-state technology in the existing computing infrastructure and analyze the costs and benefits of replacing hard drives with the NVMe drives.
Because solid state storage drives use compact electrical circuits to quickly transfer data, they can handle up to 4,000 megabytes per second, which has potential to reduce the number of bottlenecks. The downside is that their storage limit is only 500 gigabytes.
Still, the technology will complement the standard spinning-disk hard drives that have been the dominant computer storage devices for the last several decades.
With the new servers in hand, the team is exploring how to deploy the new solid-state technology in Fermilab's existing computing infrastructure.
Connect with the GCN staff on Twitter @GCNtech.