Energy buys massively parallel Linux system

Under a $24.5 million contract, Hewlett-Packard Co. is building a 1,388-processor supercomputer, the fastest single computer running open-source Linux, for an Energy Department laboratory.

Scheduled to be fully operational early next year, the Pacific Northwest National Lab's HP supercomputer will have a theoretical peak performance of 8.3 trillion floating-point operations per second and will run 30 times as fast as the Richland, Wash., lab's current 512-processor IBM SP system, said David Dixon, an associate director of the William R. Wiley Environmental Molecular Sciences Laboratory at Richland.

Installation began this summer with delivery of the first 128 nodes containing 256 64-bit, 1.0-GHz Intel Itanium 2 processors.

The production system will have 566 more dual-processor nodes based on an upgraded third-generation Itanium CPU, code-named Madison.

The nodes have an HP zx1 chip set that connects CPUs to main memory at 12.8 gigabytes/sec. HP was not the only vendor to submit a proposal, but Dixon said he couldn't discuss the other bidders. Although HP proposed Linux instead of its proprietary HP-UX operating system, hardware performance was the deciding factor, not the OS, Dixon said.

Upgrade in 2003

The first phase with McKinley processors will link processing nodes via the QsNet1/Elan3 interconnect fabric from Quadrics Supercomputing World Ltd. of Bristol, England, said Scott Studham, a group leader of computer operations.

When the rest of the processors are installed in 2003, the system will be upgraded to a Quadrics QsNet2/Elan4 fabric, which is still in development.

The fully operational supercomputer will have 1.8T of memory, 117T of local storage distributed among nodes and a 53T storage area network.

Unlike the classified parallel systems used by other Energy labs to simulate aging of nuclear weapons, the Linux supercomputer will tackle unclassified problems, such as the environmental effects of uranium waste storage at Energy's Hanford, Wash., site and the interactions of protein molecules within human and microbial cells.

The Hanford site housed plutonium production plants during World War II and the Cold War, and Energy started a massive cleanup effort there in 1988.

Researchers expect the supercomputer to reach a much larger fraction of theoretical peak performance--possibly up to 6.9 TFLOPS--than many existing massively parallel systems, Dixon said.

Better performance

"It's an awfully high percentage of peak performance, but we think we can do it," he said.

The capability will supplement or replace experiments that are "too difficult, dangerous or expensive" to perform in real life, Dixon said.

The $24.5 million contract includes HP services and support, said Martin Fink, the company's general manager for Linux systems operation. He said HP has been working to increase its presence in high-performance computing over the past 18 months, unrelated to its acquisition of Compaq Computer Corp.

IBM Corp. has been most prominent in the market in recent years. Meanwhile, Compaq is building a classified 30-TFLOPS system for Energy's Los Alamos National Laboratory.


  • 2020 Government Innovation Awards
    Government Innovation Awards -

    21 Public Sector Innovation award winners

    These projects at the federal, state and local levels show just how transformative government IT can be.

  • Federal 100 Awards
    cheering federal workers

    Nominations for the 2021 Fed 100 are now being accepted

    The deadline for submissions is Dec. 31.

Stay Connected