About to buy a firewall? Read this first

Guarding publicly accessible areas such as World Wide Web sites is as important as
protecting an internal network. So managers need something between these areas.

That's where a firewall comes in. A firewall's job is to enforce a network access
policy by monitoring all traffic going to or from the network. The firewall should let
through only the traffic authorized by the security policy and should itself be immune to
attack.

Firewalls generally follow one or more of three methodologies:

All firewalls hide internal IP addresses from external view, letting managers use any
IP address on the internal network and easing the IP address crunch while hiding machines
from attackers.

Even more than the all-important security features, a firewall's manageability and
performance are key.

The National Software Testing Laboratories Inc. put 20 firewalls through an exhaustive
evaluation that included nearly 100 forms of attack, an investigation of management
features, and a stress test over a 100-megabit/sec Fast Ethernet LAN.

Products varied widely. Security wasn't a given--several products had vulnerabilities
in their code. This was surprising, because the lab had asked the vendors to configure
their own products to avoid breaches caused by human error during testing.

Management capabilities also were diverse. Some firewalls couldn't handle the heavy
loads of the performance tests, which raises concern about just how safe things really
are.

Still, several products held their own in security, manageability and performance:
Firewall-1 from CheckPoint Software Technologies Inc., Cyberguard Firewall from Cyberguard
Corp., Watchguard Security Management System from Seattle Software Labs Inc. and Sunscreen
EFS from Sun Microsystems Inc.

The beta version of AltaVista Firewall 97 3.0 from AltaVista Internet Software excelled
in management ease.

An encouraging trend emerged in products designed for Microsoft Windows NT. Many
organizations see NT as unfit for the enterprise, but NT entries fared no worse in
security or management than many of the Unix products.

Both Unix and NT can secure sensitive data yet have weak points. If a hacker can crack
the Unix root user privileges, the entire network and its resources are at the hacker's
mercy.

NT, besides having an equivalent of the Unix root user, stores passwords in clear text
in the system registry. Even if a firewall limits external access, hackers can exist
within an organization. A system registry with readable passwords poses an easy target.

Security weaknesses often result as much from improper configuration as from inherent
operating system design. Systems administrators should identify as many holes as possible
and configure the network to be secure. Many firewall vendors offer software with
specially modified operating systems.

For security scanning, NSTL used Safesuite from Internet Security Systems Inc. of
Atlanta to launch nearly 100 attacks against each firewall. Some attacks exploited known
weaknesses in firewall or OS code. Others were denial-of-service attacks that tried to
halt the firewall operation or minimize protection.

NSTL configured each firewall with three interfaces: one for an internal network, one
for an external network such as the Internet and one demilitarized zone (DMZ) with Web and
File Transfer Protocol servers.

The DMZ added an extra measure of internal protection: Even if an attacker on the
external segment managed to compromise machines on the DMZ, everything inside remained
guarded by the firewall.

Not all vendors support such three-way configurations. Products from AltaVista, ANS
Communications and Network-1 Software and Technology Inc. work only in a dual-homed
configuration with two interfaces. NetRoad Firewall from Ukiah Software Inc. can support
two interfaces, but the vendor configured it to work with three for the tests.

All the companies are looking to add three-way support--something AltaVista already has
in its other products. In the meantime, the vendors recommended placing a packet-filtering
router in front of public servers on external segments. That's a good idea even for
three-way configurations.

Some of the firewalls had holes. Safesuite categorizes problems it finds into high-,
medium- and low-risk groups.

Most problems were of the low-risk variety, but some of Safesuite's findings might not
represent security problems in all cases. For example, a "finger" proxy might be
running on the firewall that tells the sender not to try that stunt.

Several products had medium-risk weaknesses. Among the most serious was vulnerability
to synchronize/start flooding, known as syn flooding or synchronize storms. This
denial-of-service attack has put some Internet service providers down for a day or more.

A syn flooding attack bombards the firewall with requests to synchronize Transmission
Control Protocol connections. The firewall responds by allocating all available buffer
space to these requests and blocking legitimate connections.

The only product with this vulnerability was the Gnat Box from Global Technology
Associates Inc. Despite its medium-risk classification, syn flooding is a concern for many
managers.

Another medium-risk vulnerability we saw was TCP sequence prediction, which fools
applications using IP addresses for authentication--such as the Unix rlogin and rsh
commands--into thinking forged packets actually came from trusted machines.

Four products had this problem: Centri from Global Internet, Internet Connection
Secured Network Gateway from IBM Corp., Black Hole from Milkyway Networks Corp. and Eagle
NT from Raptor Systems Inc.

Milkyway Networks acknowledges the vulnerability and said it plans to patch it in a
forthcoming release, but the company notes that the firewall does not allow access to any
of the services that might be exploited.

A strong showing in security testing does not necessarily add up to invulnerability.
NSTL mounted well-known attacks against properly configured devices on one test bed. New
attacks are no doubt on the way, devices can be misconfigured, and firewalls may behave
differently in other environments.

Security breaches are much more often tied to improper configuration than to buggy
code. Because firewall management features have strong implications for network security,
the lab focused most of its testing in this area. There are more choices in management
interfaces than there are products.

Most of the firewalls tested have graphical interfaces, though a few vendor
representatives suggested a command-line interface is faster. Notably, we found IBM's
character-based interface faster and easier than some vendors' graphical interfaces. Many
products have graphical client software for remote configuration.

Because of the range of interface options, comparing the products directly was
impossible. Instead, NSTL devised five typical firewall scenarios and rated each product
on its ability to handle them, on a scale of 5 for excellent down to 1 for poor.

Firewalls were rated only for inherent features. No credit was given for features
requiring external hardware or software, such as writing your own Unix shell script to
parse a log file and mail the results.

Scenario 1 required a firewall to support remote configuration. Scenario 2 looked at
attack tracing ability. Scenario 3 covered access to multiple Web servers. Scenario 4
subjected each firewall to four types of denial-of-service attacks. Scenario 5 tested
report and alert setup.

As WAN links get faster and intranets spring up, users demand speedier firewalls.
Sensitive network resources that previously weren't visible on IP backbones, such as
payroll or personnel servers, now need protection. And these resources probably reside on
segments running at 10 megabits/sec or faster.

To make the evaluation reflect these trends, NSTL tested most firewalls with
100-megabit/sec Fast Ethernet interfaces. The lab tested a couple of products that don't
support Fast Ethernet on standard Ethernet. It used 100Base-T switches to ensure that each
firewall had a true 100-megabit/sec load, unfettered by collisions. It verified that the
offered load ran at about 98 megabits/sec.

Admittedly, only a fraction of firewalls today have Fast Ethernet interfaces. The most
common configuration still has one or more 10-megabit/sec LAN interfaces and one
1.5-megabit/sec T1 or slower interface on the WAN side.

But a 100-megabit/sec test shows absolute performance, regardless of firewall speed.
For example, a firewall with maximum throughput of 3 megabits/sec runs at that rate
regardless of whether it connects 10- or 100-megabit/sec segments.

This performance testing caused vendors the most worry. They didn't seem comforted when
the lab told them it was third in importance on our list. Then we discovered why.

Simply put, some firewalls just aren't ready for service on 100Base-T production
networks. Many vendors had problems getting their products to work at all, and more than
half had to replace their drivers or patch firewall code.

The problems cut across OS lines. Unix and NT products had problems ranging from slow
or erratic performance to system halts. These are alarming results in terms of performance
and security.

But some products managed to shine. In handling offered loads of up to 100
megabits/sec, firewalls from AltaVista, ANS Communications, CheckPoint, Global Internet
and Sun Microsystems moved traffic at average rates of 50 megabits/sec or faster for up to
48 virtual clients.

This 50 percent performance hit might seem severe, but bear in mind that collisions on
shared-access networks will limit real-world data rates to 40 megabits/sec at most. The
CyberGuard and NetGuard firewalls also moved data at rates above the real-world maximum.

Throughput for most products kept climbing for up to 48 virtual clients, but all the
top performers took a nosedive when fielding requests from 64 virtual clients. Performance
dropped from between 28 percent for CheckPoint--which already was declining from its peak
level of 32 virtual clients--to a whopping 64 percent for NetGuard.

Secure Computing's Sidewinder did not complete the tests with 64 virtual clients. The
only firewalls that didn't blanch when faced with a load of 64 virtual clients moved
traffic well below the 10-megabit/sec mark.

These results seem to settle one of the debates raging in the firewall community:
whether Windows NT is ready to handle the heavy traffic levels on backbones. NSTL's
results suggest that it is. One of the fastest firewalls in this test, Global Internet's
Centri, ran under Windows NT 3.51 with a modified version of the NT TCP/IP stack.

On the other hand, it's possible that the version of NT has some effect on performance.
The other NT entries--from Raptor and Network-1--used Microsoft's Windows NT Server 4.0
and an unmodified version of Microsoft's IP stack.

AltaVista Internet Software's AltaVista Firewall 97 was one of the strongest
entries. Its well-designed Web interface offers predefined security policies to help
managers get started. Once the system is up, a traffic light icon displays at-a-glance
indicators of key parameters. AltaVista's firewall also was one of the fastest. Its
shortcoming is that it supports only two interfaces.

ANS Communications' ANS Interlock 4 firewall boasted fast performance and solid
management tools, especially when alerting managers to security threats. Interlock also
has the best built-in tools for managing log data. But the product as tested had only two
interfaces, ruling out DMZ segment use for public hosts.

CheckPoint Software Technologies' Firewall-1, a Reviewer's Choice, had
cross-platform support, airtight security, good performance and a rich but simple
graphical user interface, which together have made it the market leader. CheckPoint
developed the stateful inspection architecture, which lets the firewall safely transport
virtually any application.

Version 3.0 will add ActiveX screening, virus scanning and tools for managing multiple
firewalls. Although Firewall-1 tailed off relatively early in performance tests, it moved
the heaviest loads at rates faster than those of most real-world Fast Ethernet segments.

CyberGuard Corp.'s CyberGuard Firewall 3.0, also a Reviewer's Choice, posted
virtually flawless results. It was the only product with a perfect score in
denial-of-service tests and the only one to shut down external access when log files
filled--a safer course than overwriting old log entries or working without logs.

It also offered centralized management and load balancing for sites with multiple
firewalls. CyberGuard Firewall runs on secured versions of AT&T Corp.'s or Santa Cruz
Operation Inc.'s Unix operating systems.

Cycon Technologies' Labyrinth had solid management tools. It was one of the few
products that automatically e-mailed alerts in response to specific events in log files.
It moved traffic at 15- to 30-megabit/sec rates, though performance tailed off sharply as
the load increased.

Labyrinth's prices of $4,295 for 100 users and $8,695 for 1,000 users were among the
lowest of the vendors.

Global Internet's Centri Security Manager not only had security features not
found in most Unix firewalls, such as screening for ActiveX content, it also proved one of
the fastest performers. It owed this to an IP stack the vendor created by modifying
Microsoft source code. But Centri wasn't flawless on the security front, proving
vulnerable to TCP sequence-prediction attacks. Like the other Windows NT firewalls, Centri
runs on an unmodified version of the NT kernel.

Global Technology Associates' Gnat Box is a clever idea--a firewall on a floppy
disk--that's poorly executed. Besides its vulnerability to syn flooding, the Gnat Box
offered only bare-bones management capabilities.

Remote reconfiguration was impossible, although the device requires an external station
for logging, which enhances security.

IBM's Internet Connection Secured Network Gateway has extensive management
tools, decent performance and flat-rate prices for large sites. The character interface of
this AIX firewall makes it easy to set up and save configurations. IBM plans to release an
upgrade with a Java graphical interface.

Milkyway Networks Corp.'s Black Hole is strong on management and comes with a
graphical interface for easy definition of complex filtering rules. Security testing
turned up several false positives. The only legitimate vulnerability was for TCP sequence
prediction, and the vendor said it doesn't allow access to any services that might be
exploited through this weakness.

A bigger concern was performance. Black Hole completed testing only when offered 50
uniform resource locators. All the other products handled 100.

NEC Technologies Inc.'s PrivateNet Secure Firewall Server management functions
are still carried out through a character interface. The current release supports the
Socks Plus security protocol, an upgrade to the Socks protocol that NEC helped develop.

But NSTL couldn't configure PrivateNet to handle multiple Web or FTP servers on the
same segment, even though the company claimed support for this feature. Problems with the
Fast Ethernet drivers for the Berkeley Unix firewall forced the lab to run performance
tests on 10-megabit/sec segments instead.

NetGuard's low-cost Guardian 2.0 firewall has big-ticket features such as
stateful inspection and suspicious-events detection. One security concern: Vulnerability
to a denial-of-service attack mounted as logs filled up. The firewall kept working but
wouldn't log events.

Guardian performed well until the lab hit it with the heaviest load, when throughput
fell by nearly half.

Network-1 Software and Technology Inc.'s Firewall Plus for Windows NT does
stateful inspection but looks at all layers of all packets, so it can filter for virtually
any protocol. The product couldn't be clearer about defining trusted and untrusted zones.
Everything gets a devil or angel icon. The management interface was among the best we saw,
but the product offered only two interfaces. Network-1 is working on a DMZ interface.

Raptor Systems Inc.'s Eagle NT 4.0 was tightly integrated with NT's management
tools and one of the easiest products to use. On the security front, however, the
Safesuite scan turned up possible vulnerability to attacks using TCP sequence prediction.
That may not be an issue for NT shops because NT doesn't run apps that can be exploited
through this vulnerability.

EagleNT proved itself a modest performer, moving traffic at around 16 megabits/sec.

Seattle Software Labs Inc.'s low-cost, easily configured, full-featured WatchGuard combined
the major approaches to firewall design with some things lacking in most other products:
inspection of executable content such as Java or ActiveX, and the ability to
e-mail trace-route or finger information to managers.

WatchGuard was also a Reviewer's Choice.

On the downside, WatchGuard supports a limited number of proxies, and NSTL was unable
to test it with 100-megabit/sec interfaces.

The firewall runs on a modified version of Linux, but it can be configured from Windows
95 or NT. Setup is through a Windows-style configuration wizard.

Secure Computing Corp. was the only vendor with two entries, Borderware Firewall
Server 4.0.1 and Sidewinder Security Server 3.0.1. Sidewinder was the stronger
product. It posted one of the best showings in management testing and had good tools for
massaging reams of log data. Sidewinder was one of the few products to warn managers about
potential configuration disasters.

The Borderware entry was more problematic. It couldn't accommodate multiple FTP or Web
servers on the same segment, and it was among the slowest performers--largely attributable
to the 100-MHz Pentium platform supplied by the vendor.

Sun Microsystems Inc.'s SunScreen EFS, a Reviewer's Choice, was not only the
fastest product but also one of the easiest to use. The Solaris SunScreen owes its
top-notch performance to stateful inspection and special caching. The graphical interface
made short work of defining custom access rules, so virtually any IP application could be
handled. For sites with multiple firewalls, SunScreen has good tools for remote and
centralized management.

Technologic Inc.'s BSD Unix Interceptor was the only firewall with built-in
tools to e-mail or page managers in response to certain security threats. It gave good
control over logged events. But performance dropped sharply when NSTL hit Interceptor with
the heaviest load.

After the lab had pushed Interceptor hard, even lighter loads moved more slowly than
before, raising denial-of-service concerns. The relatively complex configuration needed
for multiple Web or FTP servers on the same segment also was a problem.

Trusted Information Systems Inc.'s Gauntlet has been around longer than most
commercial firewalls, so it handles more services and more kinds of customization than
almost any other. It was one of only three in this test to support screening of Java or
ActiveX objects. Still, we found Gauntlet tougher to configure than many other products,
and its performance results were disappointing.

Even running under Solaris on a Sun Ultra server, the best-case throughput was only 11
megabits/sec on a 100-megabit/sec segment, and that dropped to just 3 megabits/sec under
the heaviest load.

Ukiah Software Inc.'s NetRoad should interest Novell NetWare shops for Internet access.
NetRoad combines a firewall and an IP/IPX gateway. Configuration and management chores are
completely integrated into the Novell Directory Service tree structure. An NT version is
in beta testing.

National Software Testing Laboratories Inc., an independent organization in
Conshohocken, Pa., provides information technology testing services to developers,
corporations and governments worldwide.