By GCN Staff

Blog archive
Robot typing at a computer

Curb spear phishing? Separate bots from browsers

Sandia National Laboratories, like many government agencies, gets thousands of visitors each day searching its websites — some human-generated traffic coming through browsers, and some Web crawlers or bots that could be up to no good.

In order to protect the network, analysts have to sift the bot traffic, which can contain various threats, from legitimate human-directed browser traffic.

But even the best security system can be defeated by a gullible user taken in by a spear phishing attack, one that targets specific e-mail addresses that have something the sender wants.

Sandia computer science researcher Jeremy Wendt wants to reduce the number of visitors that cyberanalysts have to check by identifying the bots. He has developed algorithms that separate robotic Web crawlers from people using browsers, according to the lab. Wendt said he believes his work will improve security because it allows analysts to look at the two groups separately and then identify the possible sources of spear phishing.

According to Sandia cybersecurity's Roger Suppona, the ability to identify the possible intent to send malicious content might enable security experts to alert a potential target. “More importantly, we might be able to provide specifics that would be far more helpful in elevating awareness than would a generic admonition to be suspicious of incoming e-mail or other messages,” he said.

According to its Web logs, the lab said its site traffic is about evenly divided between Web crawlers and browsers. Wendt is looking for a computer that doesn’t identify itself or says it’s one thing but behaves like another, and trolls websites in which the average visitor shows little interest.

Some of the differences between bots and browsers include:

Range: Crawlers tend to go all over; browsers concentrate on one place, such as jobs.

Volume: When bots try to index a site, they pull down HTML files far more often than browsers do.

Identification: Browsers often give their browser name and operating system  information. Crawlers identify themselves by program name and version number.

Behavior: Browsers go after only one page but want all images, code and layout files for it instantly, or as Wendt calls the behavior, "bursty." Bot requests, on the other hand, are not bursty, and none of the bots identified had a high burst ratio.

Now Wendt needs to bridge the gap between splitting groups and identifying targets of ill-intentioned e-mails. He has submitted proposals to further his research after the current funding ends this spring.
“The problem is significant,” he said. “Humans are one of the best avenues for entering a secure network.”

Posted by Susan Miller on Feb 19, 2013 at 9:39 AM


  • FCW Perspectives
    human machine interface

    Your agency isn’t ready for AI

    To truly take advantage, government must retool both its data and its infrastructure.

  • Cybersecurity
    secure network (bluebay/Shutterstock.com)

    Federal CISO floats potential for new supply chain regs

    The federal government's top IT security chief and canvassed industry for feedback on how to shape new rules of the road for federal acquisition and procurement.

  • People
    DHS Secretary Kirstjen Nielsen, shown here at her Nov. 8, 2017, confirmation hearing. DHS Photo by Jetta Disco

    DHS chief Nielsen resigns

    Kirstjen Nielsen, the first Homeland Security secretary with a background in cybersecurity, is being replaced on an acting basis by the Customs and Border Protection chief. Her last day is April 10.

Stay Connected

Sign up for our newsletter.

I agree to this site's Privacy Policy.