Cigital

I Repeat Myself When Under Stress, I Repeat Myself When Under Stress

Posted On February 17, 2010 by gem

Apparently the time has come to re-release the SANS/CWE 25 — something that we can expect annually. The good news is that exercises like this do plenty to hype up software security and its importance. In fact, in many ways the target of these lists is “the reporters who cover software security.” So hype = good.

So why am I not a big fan of these lists? Well, I wrote that down a year ago and what I said then still applies. Sure would be nice to see a reasoned response to my criticisms instead of repetition of the same tired ideas. If you haven’t had a chance yet, go read my January 2009 informIT column “Top 11 Reasons Why Top 10 (or Top 25) Lists Don’t Work.”

There are some important improvements in this year’s top25 list that have been discussed on sc-l. But there is also a problem that really bothers me. The SANS guys are trying to tie the top25 list to software liability (?!) and apparently think they can hold developers accountable for their bugs..well, 25 of them at least. I think this is a wrongheaded approach to software security. I would much rather talk about the real progress the field has made than to hype up yet another list and make the list a critical aspect of software contracts?! Can you imagine what such a move (if it succeeded) would do to the price of software and to the hourly rates of developers? Developers would be compensated like lawyers!

Top-n lists do have their place. In the BSIMM we note 10 firms (of 30) who follow activity [CR1.1]. Here is the activity cut from the BSIMM:

Create a top N bugs list (real data preferred). The SSG maintains a list of the most important kinds of bugs that need to be eliminated from the organization’s code. The list helps focus the organization’s attention on the bugs that matter most. A generic list could be culled from public sources, but a list is much more valuable if it is specific to the organization and built from real data gathered from code review, testing, and actual incidents. The SSG can periodically update the list and publish a “most wanted” report. (For another way to use the list, see [T2.2] Create/ use material specific to company history.)

In my view, a tailored top-n bugs list is way more useful than a generic “world list” like the SANS/CWE25. To think about why this is, consider the differences between code bases from Intel, Microsoft, Symantec, and Nokia (not to mention Wells Fargo)…all BSIMM participants. Whose bugs do you want to eradicate? Yours? Or your neighbors?

Press coverage of the “controversy”:

• SANS Institute, MITRE release new top 25 dangerous coding errors list, TechTarget.
• Top 25 Programming Errors: Should Software Developers be Liable?, Bank Info Security.
• Hold vendors liable for buggy software, group says, ComputerWorld.
• 25 ways to better secure software from cyber attacks, Scientific American Observations.
• Security agencies release Top 25 programming errors, Washington Technology.
• Proposal Would Hold Software Developers Accountable For Security Bugs, Dark Reading.
• Group Proposes Suits Over Faulty Code, Gov Info Security.