NIH finds power in numbers

Scientists at the National Institutes of Health’s National Heart, Lung and Blood
Institute watch very small events that happen very fast.

“We study the quantum chemistry of systems,” said Eric Billings, a staff
scientist in the Computational Biophysics Section. That means tracing the movements of
individual atoms at picosecond, or 10-15-second, intervals.

One of their tools is the Chemistry at Harvard Molecular Mechanics simulation program,
called CHARMM, developed by section chief Bernard Brooks.

“We run the simulation billions of times to get a reasonable estimate of
what’s happening,” Billings said. “We’re always looking for additional
computer time. For the next 10 or 15 years, we’ll be able to saturate any system we
can get.”

Until recently, the scientists had to wait in line for time on supercomputers, such as
NIH’s IBM Corp. SP2 system. They considered stacking up lots of smaller boxes on a
fast network to get almost the same performance at a fraction of the cost.

Billings, project head for the off-the-shelf parallel computing project, is awaiting
arrival of the first components for LOBOS II, which will have 64 Pentium II dual-processor

Billings has not measured LOBOS’ performance in terms of floating-point operations
per second, but he said it can execute in two hours the same benchmark operations that
take 1.93 hours on the SP2. LOBOS cost $182,000 compared with more than $1 million for the

LOBOS is a Beowulf-class cluster of PCs running the freeware Linux operating system.
The class is named for the first computer of its kind, built in 1995 at NASA’s
Goddard Space Flight Center.

Beowulf spawned a handful of similar projects at government and university
laboratories, including Loki with 16 Pentium Pro processors at the Energy
Department’s Los Alamos National Laboratory, Megalon and Daisy with 56 and 64 Pentium
Pros, respectively, at DOE’s Sandia National Laboratories, Alice2 with 16 Pentium
Pros at DOE’s Ames Research Laboratory, and the 128-Pentium Pro Hive at Goddard.

Supercomputer makers began building parallel systems in the early 1980s, constructing
expensive hardware connectors between multiple processors because demand had outstripped
the capacity of individual CPUs.

“For us, it’s about 10 percent,” Billings said. The scientists simply
used Fast Ethernet to make 100-Mbps connections between nodes. The cost of Gigabit
Ethernet components was much higher, and performance would have been only marginally
better, he said. A Fast Ethernet card and switching port cost about $50 and $200,
respectively. In contrast, a Gigabit Ethernet card costs about $800 and a port about

“That’s a big difference,” Billing said. “It’s clearly
economics that drives Beowulf.”

LOBOS consists of 64 200-MHz dual-processor boxes from Soft-Hard Systems of Los Angeles
plus four master nodes in the same configuration.

A Series 300 Fast Ethernet hub from Cisco Systems Inc. of San Jose, Calif., handles job
initialization and input/output. A WorkGroup Fast Ethernet switch from Foundry Networks
Inc. of Sunnyvale, Calif., breaks input and output into multiple collision domains, and
Foundry’s TurboIron Gigabit Ethernet switch helps out in testing and benchmarking.

The nodes are configured in a ring. That slows performance compared with fully
connecting all nodes, but with full connections “you end up paying more for your
network infrastructure,” Billings said.

Many CHARMM jobs can run on only 32 nodes. The current LOBOS configuration can run
multiple jobs. If one node fails, its job is pulled automatically from the queue and the
remaining nodes that work with it are reassigned, Billings said.

When prices start to fall, Beowulf-class computers will get a power boost from new
400-MHz Pentium II processors, faster buses and Gigabit Ethernet components. For now, the
NIH scientists are happy to be supercomputing on the cheap with LOBOS.

About the Author

William Jackson is a Maryland-based freelance writer.

inside gcn

  • cyber hygiene (Lucky Business/

    Cleaning up cyber hygiene

Reader Comments

Please post your comments here. Comments are moderated, so they may not appear immediately after submitting. We will not post comments that we consider abusive or off-topic.

Please type the letters/numbers you see above

More from 1105 Public Sector Media Group