How USDA took advantage of breaches to improve security
- By William Jackson
- Jul 19, 2011
The Agriculture Department turned a problem into an asset three years ago when security breaches provided a business case for a new security operations center.
“We did not have an effective enterprise-wide security operation,” said USDA CIO Christopher Smith, speaking July 19 at the FOSE conference in Washington. “We were not agile in our decision making.”
But when Smith began getting visits from three-letter agencies about sensitive USDA information that was turning up in investigations, the department’s network security became a compelling need. “I went directly to the secretary,” he said.
For network security, the devil is in the complexity
USDA is not a department that often makes headlines, but it plays a critical role in the nation’s economy and security. It has 150,000 workers in 7,000 offices around the country and in 100 foreign nations overseeing the nation’s agribusiness, helping to protect the food supply, and a global network with 200,000 endpoints and 8,000 servers.
Its defenses were focused on the perimeter. Threat intelligence was months old when it was received and it took weeks to have security decisions approved. Working with agencies such as the FBI, National Security Agency and US-CERT, Smith built a business case for a security operations center to provide near-real-time situational awareness, and obtained support from upper management within the department and from Congress.
It received $5 million in funding for fiscal 2010 and stood up the Agriculture Security Operations Center.
During a recent incident, an anomaly was discovered within the network, analyzed and a connection that was exporting data was shut down.
“Everything happened in just under four minutes,” said John McClure, ASOC program manager and advanced program engineer for Crucial Security, a division of Harris Corp.
USDA management is willing to accept a certain level of risk in responding quickly to security incidents. “We are able to shut down connections very quickly,” McClure said.
Not every agency will be lucky enough have the law enforcement and intelligence communities creating a case for it, but Smith and McClure outlined the primary challenges in network security and USDA’s response to them in a July 19 talk at FOSE in Washington.
Network security has not kept pace with the quickly evolving technologies that must be supported and the threats they face, Smith said. Static solutions, a focus on perimeter security and a failure to share information have handicapped most agencies as threats are growing more sophisticated and high-profile, high-value targets are being breached.
“It’s a consistent, persistent problem,” McClure said, characterizing it as the successor to the Cold War in its scope and severity.
The ASOC relies on a suite of point solutions to provide network visibility, which includes firewalls, intrusion detection and prevention, security information and event management systems and tools for monitoring the status and configuration of all endpoints. The department came up with a list of 130 requirements for security tools, which no one vendor could meet.
“We did not rush,” in building the best-of-breed suite, Smith said. “There is no silver bullet.”
Although the perimeter has not been abandoned, USDA focuses on the interior of the network, looking for anomalies and information that is being exfiltrated.
“There is a great payoff in looking at outbound traffic,” McClure said. “They have to take the data out or it’s no good for them.”
Watching outbound data is not a simple job on a network capable of transmitting 93 gigabytes a minute. The ASOC now captures 8 terabytes of traffic a day for analysis. “We did a lot of traffic regeneration,” McClure said, passing it through a series of tools. “We started decrypting a lot of that traffic; we do a lot of deep packet inspection.”
As important as the security tools in the ASOC are the changing attitude of the department’s upper management and the relationships being developed with outside organizations. A willingness to take risk in responding to security events and delegate authority to the ASOC has speeded response time greatly when problems are identified. And the threat intelligence USDA receives from organizations such as US-CERT is now fresher.
“They are getting better at sharing with us,” McClure said, although, “we still have a way to go with that.”
Perfect security is impossible, Smith said, and a key to improved security is focusing resources where they are needed and protecting the things that should be protected.
“This is one we haven’t completely figured out,” he said. “We are still, in my mind, protecting too much.”
FOSE is run by the 1105 Government information Group, parent company of GCN.
William Jackson is freelance writer and the author of the CyberEye blog.