FISMA grades: What do they mean?
Connecting state and local government leaders
The new federal computer security report card is out, and once again the grades are pretty bad.
The new federal computer security report card is out, and once again the grades are pretty bad. And once again it is hard to say just what they really mean.
The report card is issued each year by Virginia Rep. Tom Davis, ranking Republican on the House Government Oversight and Reform Committee. Davis gave the 24 executive branch agencies covered in the report an overall grade of C- for 2006, a grade he said showed 'slow but steady improvement from past years.'
'Slow' is right, but I don't know how much improvement there is or how steady it has been. The grade had been stalled at D or D+ for the previous three years. Agencies receiving an F or an A this year are tied at eight each. Seven agencies improved their grades this year, six got worse and 10 remained the same. One major department, Veterans Affairs, didn't bother to provide a report for 2006 and so receives an 'incomplete.'
But the biggest challenge is determining just what the grades are measuring. The report card bills them as 'federal computer security grades,' but they are primarily based on compliance with the Federal Information Security Management Act. As I have said before, FISMA does not equal security. FISMA does not require secure IT systems; it requires a process for assessing, testing and managing IT security. Davis' grades are based largely on how good a job an agency is doing at inventorying, testing, certifying and accrediting its IT systems. It would be possible to test, certify and accredit all your systems and get a splendid grade even if your systems failed the tests and you were accrediting them despite their vulnerabilities.
That is not to say that agencies are doing this. Good FISMA compliance should enable an IT shop to improve its security posture. But we really don't know from the report card whether or not it is helping. Is it really reasonable to believe that Housing and Urban Development improved its security from a D+ to an A+ in one year, or that Justice can go from a D to an A-? Or that NASA could drop from a B- to a D-? That's what this year's grades show, and I have a hard time believing it.
FISMA can be a powerful tool for improving federal IT security, and the annual report card has done a good job in helping to focus attention on this subject. But I'm not sure just what the grades are measuring. I suspect it is not computer security and maybe not even FISMA performance.
A group made up of IT security vendors called the Merlin International Federal Research Consortium, surveyed federal chief information security officers about FISMA in advance of the report card. The results should probably be taken with a grain of salt ' only 30 of 117 CISOs participated, and 75 percent of the respondents said their FISMA grades were going to improve this year, so it probably wasn't a representative sample. But a couple of good ideas did come out of the report.
By a wide margin, the two greatest problems cited in FISMA compliance were funding and ambiguity in the way FISMA requirements are written. When asked for suggestions on how to improve the act, the CISOs didn't say anything about funding for security. They apparently do not think that is ever going to happen. But they did say that there should be better guidance to agencies for the yearly security controls tests and that FISMA guidelines should be clarified.
Maybe these two simple improvements could result in some real progress in both FISMA compliance and IT security.
NEXT STORY: USDA has data breach