My previous post sparked comments from Mike Rothman, Alex, Christofer Hoff, Arthur, and perhaps others I haven’t seen. I sincerely appreciate everyone’s considered feedback. In this case, the feedback was to tell me I’m off-base on terminology, and that’s all good. I’m happy to take lumps when I mess something up.

I really meant it when I said, perhaps unclearly, that I was talking about the meaning of information security “risk management” as the activity occurs in most organizations today, not any textbook definition of risk management and certainly not any definition that uses words like “exact” or “right.”

Is that better definition and more precise activity a good North Star? Absolutely. Do you tell an organization that is just learning and applying a subjective skill that if they don’t do it perfectly, they’re doing it wrong? Do you tell an organization that is doing something that works for them that they’re doing it wrong because it doesn’t match the textbook? Not if you actually want to be helpful.

The crux of the following screed is that purist “risk management” and the information security “risk management” that is (not could or should be, but is) practiced in the day-to-day business world are currently distant cousins that may meet in the future. I also feel this is the perfect time to trot out the old Yogiism that, “In theory, theory and practice are the same. In practice, they’re not.”

Many of the fundamental information sources that organizations are turning to today give no prescriptive guidance at all on risk management. The information security drivers don’t help much either. By way of example, the PCI audit procedures recommend that reviewers “Verify that the information security policy includes an annual risk assessment process that identifies threats, vulnerabilities, and results in a formal risk assessment.” FFIEC guidance for financial institutions desires a “multidisciplinary and knowledge-based approach” that identifies “all reasonably foreseeable threats.” This gives organizations no guidance at all and, after they have caused problems, all threats are reasonably foreseeable.

In my opinion, the common definition of information security risk management in the world today is the set of activities that is preventing me from failing compliance and examiner audits, that is streamlining my security operations (for better or worse; I don’t know because I don’t know how to measure it), and that appears to be reason I am not suffering even more security breaches. Again, there are always enlightened organizations that are doing more and better, but they are the minority. There is no common yardstick, no well of enlightenment. If FFIEC, Basel II, SOX, PCI, and assorted other examiners and auditors see a recognizable security program that is producing good results, then by definition whatever these organizations are doing is good “risk management.” Likewise, if I’m not suffering breaches, then I have good “risk management” in the eyes of those whose opinion I really care about (e.g., people who buy my stuff, can fine me, or can put me in jail). Call that “information security” if you want (and so it is), but saying it’s not risk management is just po-tay-to, po-tah-to.

I have my understanding of the textbook and “perfect world” definitions of risk management. I usually cast risk management as the process that starts with risk assessment. This, of course, is a catch-all term for analyses of business goals, threats, assets, event frequencies, downsides, and so on. We also need information on vulnerability, risk appetite, internal and external mandates, and so on. Some good risk models applicable to the target environment help, too. Risk management as a process then works through decision-making and continues with activities directed at risk avoidance, risk reduction (whether through reduction of loss frequency or loss severity), risk transfer, and explicit risk retention. Lather, rinse, and repeat in an ongoing “it’s a journey, not a destination” sort of way.

“Proper risk analysis,” as Alex put it, “can’t mean unnecessary controls.” I interpret this the same as I would interpret that “proper” medical care can’t produce incorrect diagnoses and “proper” maintenance can’t ever damage a vehicle. In other words, there’s no such thing in any practical sense. Smart people trying to do the right thing with the best of intentions and the best of education mess these things up all the time. Even at Harvard, someone graduates last in each and every class.

“Proper” rarely happens in the field. Proper risk management costs too much and requires to many propeller-heads to make it go. Anyone doing “proper” (full, complete, accurate, perfect) risk management would know enough not to spend the money it takes to do “proper” risk management. While accurate conclusions are crucial, very few, if any, information security decisions require that much effort or that much precision. With all due respect, I’ll wait while that sinks in.

Yes, I suppose I am dealing with some aspect of “issue management,” to throw yet another term at those activities organizations engage in to ensure their networks and applications are appropriately secure. The “issue” is “I don’t want to overspend, but I also don’t want to be the last one to do some security thing because I don’t want to explain to a jury why I’m the only organization not doing something that might have prevented a breach.” Further, the “risk tolerance of the data owner” is often of little consequence since data aggregation that increases sensitivity, public perception, and a variety of other business problems often take “risk” decisions out of the data owner’s hands.

With respect to “If your definition of risk is correct, and if your analysis is good, then all that is left is for the decision maker to figure out how willing they are to lose money,” this doesn’t happen. Even with terabytes of actuarial data, insurance companies can’t make perfect longevity predictions and odds-makers can’t reliably predict the Super Bowl winner in October. And these are really smart people, with really cool technology, with millions of dollars in backing, and the opportunity to make billions off a good decision. In other words, they’re *really* motivated. Does anyone really believe the average CIO and CISO stand a chance of having a “correct” risk definition and “good” analysis?

As to the idea that you can’t overspend because “the data owner expends only the amount of resources they are willing to allocate to reduce possibilities to their acceptable level,” well, again, I’m stymied. There’s no response for such Pollyanna thinking. It’s like a having a doctor from a world-class hospital telling a M*A*S*H surgeon to just “do it clean, quiet environment and it’ll be okay.” For example, what if the data owner postulated above is willing to spend $1B on a $1M problem? Exactly.

Activities correlating to “done correctly” and “can’t overspend” and “perfect data” and so on only happen in textbooks and fabricated business school case studies.

Information security risk management in the field is an ugly business. It’s not a matter of “when done properly” — it isn’t. It uses gut instinct for fuel and eats academics for lunch. Sometimes the process is too ethereal and we get bad decisions and too few controls. Sometimes the process is too grounded in activity and we get bad decisions and too many controls instead of more thoughtful analysis. Sometimes, bad things like public data breaches never happen even to the goofiest organizations and, sometimes, bad things seem to happen over and over even to organizations with thoughtful, resourced, and active risk management processes. It happens.

To address Hoff’s contention, I’m not “making excuses” for it; I’m documenting it. Clearly information security and risk management are not the same thing. And, to a geologist, sculptor, or someone refinishing a kitchen, marble and granite are also very different things. To the average person needing to prop up a wobbly business process, however, they’re both just rocks.

“Risk aware” organizations do spend better, but they still end up buying the same basic information security technologies as everyone else because that’s all there is. Where I see them spending more pragmatically (the perfect word here) is in training and in configuration and use of technology, which is truly a good thing. These are everyday blocking and tackling activities at most organizations. Where risk awareness truly benefits an organization is in creative areas such as software development. Thoughtfully applied security in the SDLC of an application-driven organization (think ISV or large brokerage) will result in much more overall net goodness than another internal firewall or blocking ZIP files at the Internet boundary.

“True risk management” is indeed a wonderful tool and I will always push organizations in that direction. The progression from raw art to even a glimmer of science is still a long road for information security risk management.

As to a “proper” definition of “risk management,” I spent years bemoaning the change in the definition of “hacker.” It was once a really useful word that described a true artist with code and equipment. Now, it describes any many manner of computer-based criminal activity. I wish you better luck preserving a classic definition of “risk management.”

Technorati Tags: risk management

This entry was posted by Sammy Migues on Monday, November 5th, 2007 and is filed under Risk Management. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.