On January 12th, the CWE/SANS Top 25 Most Dangerous Programming Errors list was released. Sean Barnum (a Principal Consultant) participated in the creation of the list, and I did some off the record review myself (not for attribution).

There are some important good things about top ten lists that are worthy of mention. The notion of knowing your enemy is essential in security (as it is in warfare), and top ten lists can help get software people started thinking about attacks, attackers, and the vulnerabilities they go after. These days almost any attention paid to the problem is good attention, and the fact that the technical media is paying attention to software security at all is a good thing. Top ten lists help in that respect.

But I have some serious concerns about these kinds of lists that I wrote about in my informIT article this month:

Top Eleven Reasons Why Top 10 (or Top 25) Lists Don’t Work

Here are the reasons, stripped of history and commentary which you can find in the article:

1. Executives don’t care about technical bugs.
2. Too much focus on bugs.
3. Vulnerability lists help auditors more than developers.
4. One person’s top bug is another person’s yawner.
5. Using bug parade lists for training leads to awareness but does not educate.
6. Bug lists change with the prevailing technology winds.
7. Top ten lists mix levels.
8. Automated tools can find bugs–let them.
9. Metrics built on top ten lists are misleading.
10. When it comes to testing, security requirements are more important than vulnerability lists.
11. Ten is not enough.

This entry was posted on Tuesday, January 13th, 2009 at 4:24 pm and is filed under General Interest. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.