GCN Lab Review | E-mail appliances: How we evaluated them
The GCN Lab partnered with Spirent Communications of Calabasas, Calif. to test anti-spam and antivirus network appliances. Spirent (http://www.spirentcom.com) makes a line of equipment for testing network devices and systems.
For our tests, Spirent provided two pieces of equipment: the Avalanche, which is able to generate and send large volumes of e-mail traffic, and the Reflector, which simulates almost any size network. When testing network appliances, we place the product between the Avalanche and the Reflector and measure its performance.
For this review, we built a simulated government network of 300 users. We subjected each network appliance to three performance tests. For each one, we conducted several tuning tests to normalize the appliance. If accuracy rates were within 0.5 percent on two separate tuning tests, we then conducted one final test that produced the product's actual performance score. The final test also had to fall within the 0.5 percent margin.
For the first test, we sent a stream of spam e-mail through each appliance. We sent the spam from a pool that was not more than two days old and came directly from several drop boxes on the GCN Lab network. These drop boxes receive an average of 80,000 spam e-mails per week.
The test simulated a real-world environment and ended after 1,000 messages were sent through the appliance.
The second test mirrored the first, but used a pool of several thousand viruses, which were delivered as attachments to normal mail.
The final test was a mixed stream of good mail that we created, spam and viruses. We checked spam queues to see if any of the good mail was falsely quarantined.
We used each product's default settings for spam and virus protection. Higher scores may be achievable by tweaking the devices, though in our experience this also leads to higher false-positive scores.
For a list of other tests Spirent has conducted or helped conduct involving government, visit the company's federal page, www.spirentfederal.com.