CYBEREYE—Commentary

List creates software security squabble

Is the CWE Top 25 list a silver bullet for software security? No. But it is ammunition.

The release earlier this month of a consensus list of the most serious programming errors to be avoided has garnered quite a bit of attention, some of it predictably negative. Bloggers who are amusing themselves by dissing the effort seem to be missing the forest for the trees. They dismiss the list because it is not an absolute and perfect solution to software security, and ignore the benefits it might provide.

Development of the list, available online, was managed by the Sans Institute and Mitre Corp. with support from the National Security Agency and the Homeland Security Department’s National Cyber Security Division. It represents a consensus of the most significant errors on which the IT community should concentrate. The idea is that an industrywide consensus, culled from the more than 700 errors detailed in the Common Weakness Enumeration database, can be used to standardize requirements for software procurements, to prioritize remediation of legacy applications and to help educate coders.

The detractors are unhappy essentially because no Top-N list is all-inclusive. The whole idea of these lists is that some things get left out, and that upsets some people.

“Security is a big deal, it’s not a list,” says Gwyn Fisher, chief technology officer of Klockwork in his Klocktalk blog. Yes, security is a big deal. But Fisher makes a big assumption in declaring that “what’s outside that list is just as important as what made the cut.” The compilers of the most recent list, which represents a broad range of the people in the IT community, apparently disagree. They decided that what is inside the list is more important.

Are they right? That is open to argument. But to summarily dismiss the effort simply because the list included some elements and excluded others is unfair. That’s the nature of a list.

Gary McGraw, chief technology officer of Cigital, published in his blog the “Top 11 Reasons Why Top 10 (or Top 25) Lists Don’t Work." He basically argues that what he calls “bug parades” focus too narrowly on a set of flaws and obscures the broader issues of security and good coding. “But lists change with the prevailing technology winds,” he says, and often are based on misleading or bad metrics. “Using the CWE/SANS top 25” or any other top list “to drive your software security initiative will be a major mistake.”

Maybe. I suppose it depends on what you mean by “driving your software security initiative.” If you mean using the list exclusively and ignoring other tools, techniques and processes, then yes, that would be a mistake. If you mean taking advantage of a standardized list of a manageable size to prioritize some efforts, I don’t think that is a mistake at all.

The most fatal programming error would be to believe that the CWE/SANS list is a cure-all for software security. It is not. The folks at SANS may be guilty of overselling the list when it was released, saying, “it is going to change the way organizations buy software, right away,” and suggesting that it provided a whole new class of tools not previously available. But that does not mean it will not be a useful tool.

Fisher bemoans that fact that serious hackers are capable of exploiting more than just the top 25 errors, and that eliminating the top 25 will deter only "script kiddies" and "ankle-biters." “All this money being spent on what? A barrier to deter script kiddies. ...”

Of course, any Top-N list is like a rail fence: Once you remove the top rail, you find another top rail immediately underneath it. But serious hackers are like everyone else. If you leave the well-known, easy-to-exploit flaws in programs, they will continue to pick this low-hanging fruit. If you remove the top 10 or 25 flaws, a new top list will appear, but it will force the bad guys to refocus their efforts on new areas that might not be as easy to exploit.

Security is a big deal. And it is a never-ending process that no list will be able to halt. But there is no reason we should not use all the ammunition we can get just because a bullet doesn’t happen to be silver.

Reader Comments

Tue, Apr 20, 2010 smith

William Jackson, You are SPOT ON! Thanks for sharing such valuable information, the best computer security organizations in the world are which regularly updates and train's people for security related jobs. It imparts computer security skills to get rid of the latest security threats looming on every computer assets and also its valuable data.. For more information go through the link:http://www.eccouncil.org/certification/why_choose_ec-council_certifications.aspx

Wed, Jan 28, 2009 amrita@web hostingdubai http://www.canadahitech.com

hi nice Top 25 list a silver bullet for software security.this list is really useful to learn a software security thanks

Tue, Jan 20, 2009 Eirik Virginia

So, the list is imperfect and is not all inclusive. It has still helped with awareness. Programming flaws are the fundamental reason we have malware attacks and require patch management, Anti-Virus, Anti-spyware, and others. Microsoft Patch Tuesday reminds us every month that there were flaws six months ago unknown to us and there will be other flaws unknown to us in six months. http://www.securitynowblog.com/endpoint_security/microsoft-patch-tuesday-reminds-us-how-vulnerable-pcs-are But, signature based tools cannot protect us like they used to: http://www.securitynowblog.com/endpoint_security/secunia_report_signature-based_antivirus_misses_most_unknown_malware We need other tools but they're complex and difficult to evaluate: http://www.securitynowblog.com/endpoint_security/hips-security-software-trial-necessary-difficult

Tue, Jan 20, 2009 Dismayed

I was somewhat dismayed by this list when SANS first announced they were doing it, and now that I've seen it I still am. I'm dismayed because I don't know why it was even done. OWASP has long been considered a community of subject matter experts in the field of secure programming because they are made up of programmers, not PhDs. OWASP actually dropped some of the items SANS felt necessary to place on the list (yes OWASP did use a standardized CVE too if I recall). So, ultimately I'm dismayed that, especially while our nation sees a tightening of the belt, our federal government saw fit to colloaborate not with the experts who are doing grassroots work for free, but with an "Institute" who will most likely slap a $3,000 price tag on it and start training people on it. Thanks, but no thanks SANS, I like my OWASP list (as does the PCI Council) and I will continue to support them and their efforts.

Please post your comments here. Comments are moderated, so they may not appear immediately after submitting. We will not post comments that we consider abusive or off-topic.

Please type the letters/numbers you see above