Feds develop freeware app to foil hackers

Shadow combines monitoring with
statistical assessment to detect events that filters cannot decode.

A consortium of agencies and private organizations has released a free network
intrusion detector to combat about 40 types of cyberattacks.

The intrusion detection team at the Naval Surface Warfare Center at Dahlgren, Va.,
primarily developed the Shadow freeware using Energy Department code.

To help develop and test the software, the center worked with the Defense Information
Systems Agency, Defense Logistics Agency, Army Research Laboratory, Centers for Disease
Control and Prevention, Space and Naval Warfare Systems Command, Geological Survey and
DOE’s Los Alamos National Laboratory.

Shadow is available from the Sans Institute, a cooperative research organization in
Bethesda, Md. Although a Navy server hosts the code, users must obtain documentation from
the Sans Institute before downloading the software.

“We want users to know what they are doing before they download,” Sans
director Alan Paller said. Sans also presents two-day training programs in several cities.

Shadow’s developers call it the first cooperative, public-domain tool of its kind.

Intended to combat joint efforts by hacker groups, it uses TCPdump, a DOE program that
monitors and filters TCP activity, and Network Flight Recorder freeware from Network
Flight Recorder Inc. of Woodbine, Md., for monitoring, archiving and alerting tools.

The current version runs only under Unix. The developers said if an agency has large
log files, it will likely need several gigabytes of drive space to run the software.

“We have achieved relatively high efficiency,” said Stephen Northcutt, head
of the Naval Surface Warfare Center’s intrusion detection team and Shadow team

“It’s not a perfect system,” he said. “It needs to work in
conjunction with something like a network intrusion device. But if you don’t have the
money for a commercial system, or if you don’t know what you want,” Shadow is a
place to start.

Northcutt’s team, which provides network intrusion detection services at 10
Defense Department sites, began working on Shadow a few years ago.

“We built it to solve a problem we had,” he said. At that time,
Defense’s intrusion devices could monitor only five protocols for suspect strings on
20,000 DOD computers, he said, and they produced many false alarms.

“The false alarms are what kill you,” Northcutt said. “We needed
something that was cost-effective in terms of people. I had to be able to use it with
lower-paid analysts.”

The Shadow program got a boost last November when Victoria Irwin joined
Northcutt’s team. The new analyst “tore out our assumptions” and combined
monitoring with statistical assessments, he said.

Through statistical assessment, analysts using Shadow have identified three new types
of attacks.

With help from the Sans Institute, Northcutt’s team in April released a
third-generation version of Shadow for review, “and we got slammed,” he said.
“We went back with our tail between our legs” and reworked the reviewers’
recommendations into the current version.

Major help came from Olav Kolbu, system administrator for the University of Oslo in

“First off,” Kolbu wrote the Shadow team, “since the shell scripts
included in the package were pretty useless, I’ve rewritten them in Perl.”

The current version of Shadow incorporates 90 reviewers’ recommendations. It is
intended to complement, rather than compete with, commercial products, and all its code is
in the public domain, Northcutt said.

“If there is a piece of it you want, for goodness’ sake, incorporate
it,” he said.

A description of Shadow—along with instructions for downloading, installing and
running it—is available to any U.S. citizen who sends e-mail to info@sans.org with the words Shadow Description in the
subject line of the message.  

About the Author

William Jackson is a Maryland-based freelance writer.

Stay Connected

Sign up for our newsletter.

I agree to this site's Privacy Policy.