Comments on: Badness-ometers are good. Do you own one? http://www.cigital.com/justiceleague/2007/03/19/badness-ometers-are-good-do-you-own-one/ The Cigital Software Security and Quality Blog Mon, 13 Oct 2008 00:29:58 +0000 http://wordpress.org/?v=2.0.11 by: gem http://www.cigital.com/justiceleague/2007/03/19/badness-ometers-are-good-do-you-own-one/#comment-28 Fri, 23 Mar 2007 17:55:44 +0000 http://www.cigital.com/justiceleague/2007/03/19/badness-ometers-are-good-do-you-own-one/#comment-28 Interesting analogy. I think I would liken the "gate review checklist" item...what might otherwise be termed an assurance activity with an associated quality gate...more to a regular and necessary maintenance activity than to something needed while you're driving. However, if we limit ourselves to dashboard gauges, maybe code review is like the gas gauge? I love the "idiot light" idea though...have to think about this some more. With regard to whether is it better to use a badnessometer to look for really stupid mistakes than code review to look for really stupid mistakes, my intuition is that you will find more actionable results with code review than you will with security testing. In the end you should do both, and trying to determine relative value for either seems a waste of time. gem BTW, on the pen testing front, my views are not at all the same as marcus's. I was simply the interviewer. You can find the complete podcast here http://www.cigital.com/silverbullet/show-003/. Interesting analogy. I think I would liken the “gate review checklist” item…what might otherwise be termed an assurance activity with an associated quality gate…more to a regular and necessary maintenance activity than to something needed while you’re driving. However, if we limit ourselves to dashboard gauges, maybe code review is like the gas gauge? I love the “idiot light” idea though…have to think about this some more.

With regard to whether is it better to use a badnessometer to look for really stupid mistakes than code review to look for really stupid mistakes, my intuition is that you will find more actionable results with code review than you will with security testing. In the end you should do both, and trying to determine relative value for either seems a waste of time.

gem

BTW, on the pen testing front, my views are not at all the same as marcus’s. I was simply the interviewer. You can find the complete podcast here http://www.cigital.com/silverbullet/show-003/.

]]> by: Scott Wright http://www.cigital.com/justiceleague/2007/03/19/badness-ometers-are-good-do-you-own-one/#comment-25 Thu, 22 Mar 2007 01:19:51 +0000 http://www.cigital.com/justiceleague/2007/03/19/badness-ometers-are-good-do-you-own-one/#comment-25 I am trying to think of an analogy. I guess a Badness-ometer is, to software security metrics, what an oil pressure gauge is to a car dashboard. Would that make a Gate Review Checklist item such as "Code-review-completed" something akin to an "Idiot-Light"? I can't help but remember a McGraw/Ranum interview article in an IEEE publication about how, in Marcus Ranum's opinion, penetration testing is the stupidest idea he's ever heard of. Something about how ~"people who don't understand how their system works hire other people who don't understand how their system works, to simulute attacks by other people who don't understand how their system works..."~ It was a very amusing discussion, and almost convincing, if it weren't for real-world constraints on most development organizations, and the fact that no matter how good the development cycle security techniques are, there will always be holes, especially in larger systems with complex interconnected parts. All this to say, I agree that the Badness-ometer is a much better thing than a "Code-review-completed" idiot light. I am trying to think of an analogy. I guess a Badness-ometer is, to software security metrics, what an oil pressure gauge is to a car dashboard. Would that make a Gate Review Checklist item such as “Code-review-completed” something akin to an “Idiot-Light”?

I can’t help but remember a McGraw/Ranum interview article in an IEEE publication about how, in Marcus Ranum’s opinion, penetration testing is the stupidest idea he’s ever heard of. Something about how ~”people who don’t understand how their system works hire other people who don’t understand how their system works, to simulute attacks by other people who don’t understand how their system works…”~

It was a very amusing discussion, and almost convincing, if it weren’t for real-world constraints on most development organizations, and the fact that no matter how good the development cycle security techniques are, there will always be holes, especially in larger systems with complex interconnected parts.

All this to say, I agree that the Badness-ometer is a much better thing than a “Code-review-completed” idiot light.

]]>