FISMA's effectiveness questioned

Some ask if its metrics can measure security

You can measure good security, but it's not being measured today.' 'Bruce Brody, CACI INternational INc.

A year ago this month, the National Institute of Standards and Technology issued Federal Information Processing Standard 200, giving agencies a year to meet the minimum requirements for securing federal data and IT systems.

It was the third of three standards publications NIST was required to deliver under the Federal Information Security Management Act.

The House Oversight and Government Reform Committee is expected to release its 2007 report card on FISMA compliance soon and, although it won't be up to date as of this month, it's expected to give a pretty good idea of how agencies are doing. But regardless of how agencies score, some critics say the problems with FISMA go deeper.

At a recent security conference in San Francisco, a pair of security experts, one of them a former federal chief information security officer, criticized FISMA as a well-intentioned but fundamentally flawed tool.

'A lot of your money is being thrown away,' Alan Paller, director of research for the SANS Institute, told an audience at the RSA IT security conference.

The 2002 act mandates security planning for agencies, requiring a risk analysis of IT systems, and certification and accreditation of those systems.

'FISMA wasn't written badly, but the measuring system they are using is broken,' Paller said. 'What we measure now is, 'Do you have a plan?' Not whether the plan actually improves security.'

'Swiss cheese'

Too often, the plans do not improve security, said Bruce Brody, vice president of information assurance at CACI International Inc. and formerly with the Veterans Affairs and Energy departments.

'Federal systems and networks are like Swiss cheese,' Brody said. 'FISMA over five years has not helped us to be appreciably more secure.'

The speakers described the risk analysis and C&A processes as paperwork drills that let agencies comply with the letter of the law without doing anything to improve actual security. Even so, many agencies routinely receive failing grades in the annual FISMA report cards, and government as a whole has not risen above a D. Brody said he received four Fs and one C during his term in government.

Paller offered two broad fixes for the security challenge facing government. The first is to stop blaming the user for problems and require that vendors ship well-designed products that are securely configured by default. He also called for using 'attack-based' metrics in measuring security compliance. These metrics include:

  • How quickly penetrations of the system are identified
  • The length of time it takes to deploy needed security patches
  • The number of accounts remaining active after employees or consultants have left an agency
  • Whether programming teams are including errors in code
  • How quickly malicious code can be found on a system.


Brody defined five things a CIO must know about his systems to ensure security:

  • The boundaries and topologies of the interconnected enterprise
  • The devices that are connected to the enterprise and the channels they use to connect to it
  • The configuration of these devices
  • Who is accessing these devices and whether that access is authorized
  • What these users are doing on the system.

'You can measure good security, but it's not being measured today,' Brody said. Brody and Paller were hopeful that changes in FISMA could be made in the new Congress.

Reader Comments

Please post your comments here. Comments are moderated, so they may not appear immediately after submitting. We will not post comments that we consider abusive or off-topic.

Please type the letters/numbers you see above